IRC log of #zope for Thursday, 2011-05-19

*** m8 has quit IRC00:37
CIA-89menesis * r121717 (19 files in 5 dirs): Conform to repository policy00:38
CIA-89menesis * r121718 (LICENSE.txt COPYRIGHT.txt): Add license and copyright00:38
CIA-89menesis * r121719 (CHANGES.txt Remove unneeded dependencies00:38
CIA-89jim * r121720 /Sandbox/J1m/customdoctests/ (4 files in 2 dirs):00:38
CIA-89Added tests for parser.00:38
CIA-89(In doing so, realized that overriding ps2 doesn't work00:38
CIA-89and it's not worth fixing.)00:38
*** sp0cksbeard has quit IRC00:44
*** J1m has quit IRC01:11
*** sm has quit IRC01:18
*** davetoo has joined #zope01:26
*** davetoo has left #zope01:27
*** menesis has quit IRC01:30
*** gqlewis has joined #zope01:46
*** MrTango has quit IRC01:58
*** evilbungle has quit IRC02:00
*** mr_jolly has quit IRC02:07
*** lcarvalho has quit IRC02:27
*** supton has quit IRC02:27
*** lane_ has joined #zope02:28
*** CIA-107 has joined #zope02:29
*** gqlewis has quit IRC02:32
*** tiwula has quit IRC02:32
*** CIA-89 has quit IRC02:32
*** gqlewis has joined #zope02:32
*** mr_jolly has joined #zope02:50
*** daMaestro has quit IRC03:14
*** webmaven has quit IRC03:14
*** River_Rat has joined #zope03:32
*** Spanktar has quit IRC03:34
*** dayne has joined #zope03:34
*** RiverRat has quit IRC03:35
*** supton has joined #zope03:50
*** sm has joined #zope03:51
*** mr_jolly has quit IRC03:55
*** dayne has quit IRC03:59
*** lane_ has quit IRC04:00
*** supton has quit IRC04:04
*** davetoo has joined #zope04:06
*** supton has joined #zope04:08
*** supton has quit IRC04:10
*** sm has quit IRC04:12
*** mr_jolly has joined #zope04:21
*** mr_jolly has left #zope04:29
*** River-Rat has joined #zope04:37
*** River_Rat has quit IRC04:40
*** davisagli has quit IRC04:44
*** davisagli has joined #zope04:45
*** allisterb has joined #zope05:20
*** allisterb has quit IRC05:20
*** davisagli has quit IRC05:35
*** davisagli has joined #zope05:36
*** gqlewis has quit IRC05:40
*** rump has joined #zope05:41
*** gqlewis has joined #zope05:42
*** gqlewis has quit IRC05:58
*** allisterb has joined #zope06:20
*** allisterb has quit IRC06:20
*** supton has joined #zope06:27
*** supton has quit IRC06:55
*** davetoo has left #zope07:03
*** ThePing has joined #zope07:56
*** ThePing has left #zope07:56
*** ccomb has joined #zope08:21
*** hever has joined #zope08:24
*** supton has joined #zope08:39
*** digitalmortician has quit IRC08:48
*** zagy has joined #zope08:56
*** wosc has joined #zope08:56
*** __mac__ has joined #zope09:01
*** tisto has joined #zope09:02
*** supton has quit IRC09:04
*** menesis has joined #zope09:25
*** slackrunner has joined #zope09:28
*** agroszer has joined #zope09:30
*** digitalmortician has joined #zope09:36
*** avoinea has joined #zope09:46
*** alexpilz1 has quit IRC09:55
*** Wu has joined #zope09:58
*** River-Rat is now known as RiverRat09:59
*** ccomb has quit IRC10:02
*** planetzopebot has quit IRC10:08
*** shastry has quit IRC10:09
*** planetzopebot has joined #zope10:09
*** shastry has joined #zope10:10
*** rump has quit IRC10:14
CIA-107tlotze 2 * r121721 zc.buildout/ (16 files in 3 dirs): removed awareness of multiple Python interpreters, making code simpler to test and getting rid of some tests with errors10:29
CIA-107icemac * r121722 zc.ssl/ (CHANGES.txt src/zc/ssl/
CIA-107- Using Python's ``doctest`` module instead of deprecated10:29
CIA-107icemac * r121723 zc.ssl/ ( COPYRIGHT.txt LICENSE.txt): Conform to repository policy.10:29
CIA-107icemac * r121724 zc.table/ (6 files in 2 dirs):10:29
CIA-107- Using Python's ``doctest`` module instead of deprecated10:29
CIA-107- Removed deprecated slugs for ZPKG and ZCML.10:29
CIA-107icemac * r121725 zc.table/ (12 files in 2 dirs): Conform to repository policy.10:29
CIA-107icemac 1.0 * r121726 zc.ssl/ ( COPYRIGHT.txt LICENSE.txt): Conform to repository policy.10:29
CIA-107icemac * r121727 zc.tokenpolicy/ (COPYRIGHT.txt LICENSE.txt): Conform to repository policy.10:29
CIA-107icemac * r121728 zc.testbrowser/ (7 files in 2 dirs): Conform to repository policy.10:29
CIA-107wosc wosc-test-stacking * r121729 zope.component/ (NOTES.txt src/zope/component/ Add test that list valued lookups are not affected by stackable10:29
CIA-107wosc wosc-test-stacking * r121730 zope.component/NOTES.txt: Update todo list10:29
*** goschtl has joined #zope10:30
*** alexpilz has joined #zope10:34
*** fredvd has joined #zope10:39
*** humanfromearth has joined #zope10:47
*** alexpilz has quit IRC10:49
*** alexpilz has joined #zope10:50
*** sylvain has joined #zope10:52
*** evilbungle has joined #zope11:05
*** evilbungle has quit IRC11:05
*** humanfromearth has left #zope11:08
*** alexpilz has quit IRC11:19
*** alexpilz has joined #zope11:19
*** mr_jolly has joined #zope11:26
*** lukasg|4tw has joined #zope11:43
CIA-107adamg * r121731 zope.wineggbuilder/master.cfg: oops, I missed a previous commit? add ZTK 1.111:48
CIA-107hannosch * r121732 Products.ZCatalog/ ( CHANGES.txt): Prepare Products.ZCatalog
CIA-107hannosch * r121733 /Products.ZCatalog/tags/2.13.14: Tagged Products.ZCatalog
CIA-107hannosch * r121734 Products.ZCatalog/ ( CHANGES.txt): vb11:48
CIA-107hannosch 2.13 * r121735 Zope/ (doc/CHANGES.rst versions.cfg): Products.ZCatalog = 2.13.1411:48
CIA-107hannosch * r121736 Zope/ (doc/CHANGES.rst versions.cfg): Products.ZCatalog = 2.13.1411:48
*** alga has joined #zope11:56
*** TomBlockley has joined #zope11:57
lukasg|4twI've got a worker instance for use with zc.sync that dies as soon as it's trying to dispatch jobs in the queue (it just returns to the prompt when started in fg)11:57
lukasg|4twDoes anyone know where in the ZODB zc.async persists its queue, so I can clean out the offending job?11:57
*** fredvd is now known as fredvd|away12:00
*** mitchell`off is now known as mitchell`12:04
*** evilbungle has joined #zope12:04
koshsorry not something I have ever used12:08
*** JT has quit IRC12:08
*** JT has joined #zope12:09
*** eperez has joined #zope12:16
*** gqlewis has joined #zope12:16
*** ccomb has joined #zope12:24
*** gqlewis has quit IRC12:24
*** teix has joined #zope12:37
CIA-107hannosch * r121737 Products.ZCatalog/ (2 files in 2 dirs): Fixed BooleanIndex' items method so the ZMI browse view works.12:41
*** mr_jolly has left #zope12:55
*** tisto is now known as tisto|away13:19
*** j-w has joined #zope13:20
*** mejo has joined #zope13:25
mejoi'm trying to debug a zope ConflictError.13:25
mejoi already found out, that it's possible to access the related object via the hexcode in event.log13:26
mejoseveral times, i found something like 'from ZODB.utils import p64' and then 'print app._p_jar[p64(0xHEX)]' in the debug python shell13:27
mejonow my problem is, that app doesn't exist:13:27
mejoNameError: name 'app' is not defined13:27
mejoi'm running zope 2.10.13. the search results from google are all around 2007. maybe 'app' has been renamed?13:28
mejoor is there another way in zope2.10 to access the object via hexcode?13:28
*** davisagli has quit IRC13:31
*** davisagli has joined #zope13:33
betabugmejo: I think you have to import app from somewhere13:37
mejoyes, but from where?13:39
mejoor is app the module that produced the ConflictError?13:39
betabughmmm, dunno, never used this stuff... I'd look on
betabugthe ConflictError is produced when 2 request try to write to the same object13:40
betabugassuming it's a "write conflict error"13:40
betabugif it's a "read conflict error", just ignore it13:40
mejoit's a database conflict error13:41
mejoand it happens regularely13:41
mejosometimes it has unresolved conflicts.13:42
betabughmmm, what do you mean "database conflict error"?13:42
mejoand twice, the zope instance even froze.13:42
betabughmmm, you'll have to find out which kind of objects get altered by what methods13:42
mejo2011-05-19T10:42:20 INFO ZPublisher.Conflict ReadConflictError at /VirtualHostBase/http/ database read conflict error (oid 0x63, class BTrees._OOBTree.OOBTree) (16 conflicts (0 unresolved) since startup at Thu May 19 10:03:08 2011)13:43
betabugit's a "read conflict error"13:43
betabugso no worries about this one13:44
mejoand this one:13:45
mejo2011-05-19T10:43:00 INFO ZPublisher.Conflict ConflictError at /VirtualHostBase/http/ database conflict error (oid 0x65, class BTrees._OOBTree.OOBTree, serial this txn started with 0x038e6cbb81084c44 2011-05-19 10:35:30.241972, serial currently committed 0x038e6cc3019ad8aa 2011-05-19 10:43:00.376141) (18 conflicts (0 unresolved) since startup at Thu May 19 10:03:08 201113:45
betabugdunno, it doesn't say "write", but it got resolved just fine13:45
mejounfortunately some of them don't get resolved13:47
mejoand as written, the instance already froze twice13:47
mejono logs at freeze time, only lot's of ConflictErrors some minutes before13:48
betabugok, but running after the conflict errors which *got* resolved won't help you13:48
mejoi see13:49
betabugit's more likely that due to the resolving of conflict errors zope ran out of threads - and that's what you see as "frozen"13:49
mejostill i don't know how to run after conflict errors at all13:49
betabug"due" in the sense of "it takes longer to process the request"13:49
betabugyou read the code and try to see why it fails13:50
betabugfind out which objects are involved, which requests get stuck13:51
betabugand then you'll narrow it down to the methods that take so long to run and/or that alter objects13:51
mejoyes, but to find out the objects, i need to translate the OID hexcode into object name13:51
betabugI never use OID for anything13:52
mejothe event.log only mentions the oid13:52
betabugI suggest you look in the Control_Panel debug info what your long running requests are doing13:52
betabugor - if the thing is already stuck - install DeadLockDebugger and have it dump the traces13:52
mejothanks a lot for your help!13:53
betabugno problem, glad to help13:53
mejoi'll see what i find out and report back ;-913:53
betabugno problem13:54
*** ccomb has quit IRC13:55
mejosorry to ask again:13:56
mejohow long may it take, when zope runs "out of threads"?13:56
mejoand is there anything we can do about it?13:56
betabughmm, it's depending on your code13:56
betabugif you have (as is standard) 4 threads configured and you have 4 people clicking on a link that takes each 120 seconds to process, then you're "closed" for 120 seconds13:57
betabugnow you could say "we need more than 4 threads!!", but the real problem is "why does something take 120 seconds?"13:57
mejoany disadvantages of increasing the number of threads?13:57
betabugyes, increasing the number of threads doesn't solve the problem13:58
mejomaybe it's related to heavy mysql transactions13:58
betabugit just postpones it a little13:58
mejowe heavily use the mysql database adapter13:58
betabugthen you'd go and isolate them and either cache them or rewrite to find another solution (e.g. request and store, retrieve later)13:58
betabugmejo: I'm working on a similar case for a customer right now, a big, old site and they get a deadlock after a few hours14:00
betabugbut it's a different cause at the bottom :-)14:01
mejobetabug: the problem with debug info at Control_Panel is that it has no history14:05
mejoso once the instance hangs, i'm no longer able to check which connections do exist14:05
betabugonce it hangs you need DeadLockDebugger14:06
mejoin most situations, we don't have old connections at the debug info14:06
mejook, I'll take a look14:06
betabugbut you can do a cronjob to write the debug output to disk every x minutes14:06
mejothat's a good idea.14:06
mejojust fetch with wget, or is there an easy solution to access it within a python script?14:07
betabugI'd use lynx/wget/curl14:08
*** fredvd|away is now known as fredvd14:14
*** J1m has joined #zope14:16
*** J1m has quit IRC14:33
*** dayne has joined #zope14:36
koshbetabug: you know what is funny is that on my sites when I tested then at 500 concurrent requests to 4 zope servers behind nginx and about 100K requests total I got not conflict errors for reading of any kind14:42
koshand write conflict errors seem absurdly rare in all my systems14:42
betabugkosh: you have different code :-)14:43
koshI just wonder what kind of code people write that causes it so I can try even harder to avoid those issues14:43
koshone thing I found long ago though is that if I test if data is different before I write it that seriously sped up my zope apps and made conflict errors far less common14:43
koshso if two people try to write to the same object but try to write to different parts of it things pretty much just work14:44
koshbut that is also rare as hell to have happen14:44
kosha simple way I can think of to cause conflicts is to keep a log of something in a basic python object14:45
koshso you write a tuple to a list on every page view or something like that14:45
koshthe entire list would get resaved each time and be massively likely to cause conflicts and cripple the site speed wise after a while14:45
*** wosc has left #zope14:46
betabugkosh: that's the situation on one site I'm debugging14:49
betabugwe're talking a huge list, 7000 objects14:49
*** Wu has quit IRC14:52
koshchange it to an OOBTree14:53
koshif you need to keep order then use a full timestamp as a key for insertion14:53
koshthat way you won't have any collisions and also don't need to check before insertion14:53
koshand it will naturally order14:53
koshhowever I have also seen systems have huge problems when you call out to an external db with bad queries14:55
koshso if you have a few poor mysql query that takes 30 seconds to run that can deadlock things pretty fast14:55
betabugwell, I need order and I need to re-order14:59
betabugso I wrote a litte "TreeList" object14:59
betabugnow is the task to integrate it and then to move the data14:59
koshhow did you make it so you can change order and also add efficiently?14:59
betabugwell, changing order is not efficient right now15:11
betabugbut it's happening rarely15:11
betabugefficiency is the underlying btree as storage15:11
betabuginteger keys15:11
betabugso with inserts and changing order I have to renumber15:11
betabugnow thinking about migration strategies15:13
koshwell the plus side is people screw up just as badly with a relational db as they do with zope15:17
betabugyeah, they do with everything15:17
koshhave seen queries that go from hours to less then a second with properly changing the query and indexing15:17
*** dayne has quit IRC15:23
koshbetabug: if you indexed them with floats you would be able to change the number to a float between the ones on either side of what you wanted15:27
koshbetabug: that way a minimum of changes are needed to reorder15:27
betabugyeah, thought about that15:27
betabugbut it said somewhere that FOBTree is available over a certain ZODB version, so I didn't investigate15:28
betabugmaybe too lazy15:28
betabugthe funny part was learning to mimic a python list :-)15:28
koshyou could have done an OOBTree and that would have worked15:32
betabughmm, right15:35
betabugI was too lazy15:35
betabugbut inserting/reordering is happening only in 2 rare cases, so I don't mind so much15:36
betabugwhile time is a big point right now15:36
betabugthe thing I'm wondering is if I try to migrate on-the-fly or if I go through all the objects15:36
betabugon-the-fly could take a long time, with visitors giving up on those requests15:37
koshmigration though should be fairly fast15:38
kosh7000 inserts into an IOBTree shoudl be damn fast15:38
betabugyes, but the heavy object loading would still be there15:38
betabugit's what bogs the app down right now15:39
koshso very late at night when nobody is using it do a migration15:39
betabugyeah, probably15:40
*** slackrunner has quit IRC15:50
CIA-107janwijbrand * r121738 megrok.chameleon/ (3 files in 3 dirs): update changelog15:52
CIA-107janwijbrand * r121739 megrok.chameleon/ (CHANGES.txt README.txt): update readme and changelog15:52
CIA-107janwijbrand * r121740 megrok.chameleon/src/megrok/chameleon/README.txt: update another readme15:52
CIA-107janwijbrand * r121741 megrok.chameleon/ (buildout.cfg CHANGES.txt reflect the RC-ness of and Chameleon in the version number for megrok.chameleon15:52
*** dayne has joined #zope15:56
*** J1m has joined #zope15:59
koshbetabug: however you should be able to do thousands of records per second usually pretty easily16:05
koshhail evil J1m!16:06
*** dayne has quit IRC16:09
*** sp0cksbeard has joined #zope16:14
*** superdupersheep has joined #zope16:19
superdupersheepdoes anyone know if zope 2.11 is compatible with a 64 bit Python 2.4?16:20
*** pjfd4 has joined #zope16:24
*** tisto|away is now known as tisto16:26
*** davisagli has quit IRC16:29
*** davisagli has joined #zope16:30
koshsuperdupersheep: should be16:30
superdupersheepwe run a *huge* site16:30
koshsuperdupersheep: I have been running zope with 64bit python long before 2.416:30
koshso test it16:30
koshyou would not deploy to a live site anyways16:30
superdupersheepwell, it was tested, it looked good, then in production it fell apart16:31
superdupersheepeverything "works", but certain sites fall to bits after several hours16:31
koshI have not seen any difference between 32bit zope and 64bit zope on any tests and all my production systems are currently 64bit16:31
koshand those systems all started as 32bit, same database and everything16:31
koshwell what falls to bits? what errors do you have?16:32
superdupersheepno errors per se (or at least, very little in the logs) - the symptom is that Zope just gets very, very slow16:32
superdupersheepit seems to be linked to memory consumption, even though the machine has *tons* of RAM and doesn't seem to be CPU bound at all16:32
superdupersheepwhen i say a huge site i mean a very sizable university site :)16:33
J1mhuh, why am I evil?16:33
koshJ1m: no reason at all, but it is funnier the hello16:33
koshsuperdupersheep: so is the memory consumption pushing the machine into swap?16:34
koshwhy do you think it is memory consumption then?16:34
superdupersheepwell, debugging things like this with zope is basically impossible16:35
koshwhy do you think that?16:35
superdupersheepbecause i have no diagnostic tools to introspect what the hell Zope's up to16:35
superdupersheepi have the system's diagnostic tools16:35
superdupersheepthat's about it16:36
koshactually there are a lot of diagnostic tools available, you may not understand them though16:36
koshfor example if you can find out which pages are causing problems you can find out what is causing the stalls16:36
koshif it is object loads it would show up in your cache page as a large load when that page is pulled16:36
*** __mac__ has quit IRC16:36
koshif it is an external db that is stalling you would at least know which page is causing it16:36
superdupersheepi should elaborate a bit more, before you go any further16:37
* betabug bets 20c on the p-word16:37
superdupersheepwe have many 32bit machines that are in production, with the same version of python, same version of zope16:37
superdupersheepthey don't suffer from any slowdown in any way16:37
koshwhat os are you running on? how did you setup python for 64bit? did you compile all your c extensions or just copy them over?16:38
koshalso the problem could still be something else16:38
superdupersheepkosh: i'm not that stupid :)16:38
koshlike if you have a postgres server that zope talks to that could be screwed up on your 64bit system from some config error16:38
superdupersheepnope, no postgres involved.16:38
koshwell subsitute anything else for that also like mysql, db2, oracle etc16:39
superdupersheepall built as 64 bit16:39
koshcomputers are not random, if some pages are super slow and others are fine then it needs to be isolated about what those pages are doing that is so different16:39
koshmaybe a library they need was forgotten16:39
koshwhat os?16:39
superdupersheep(not using system python, obviously)16:40
superdupersheepif a library was missing you'd expect it to not work immediately, rather than work fine for a period of time, then suddenly get slow16:40
koshgetting slow after a period of times makes very little sense unless memory is running out16:41
koshhow is zope setup? do you have zope talking to zeo servers? how are your caches setup?16:41
superdupersheeptalking to zeo servers; client cache for zeo 512MB, 120,000 objects16:43
koshI assume that zeo is on another machine?16:44
superdupersheepyes. that machine isn't short on RAM, has basically 0 load averages, doesn't appear to be working hard at all16:45
*** j-w has quit IRC16:45
* kosh tries to remember how the cache flipping things worked for slowing things down16:46
koshdo you have any warnings in your error log about cache flipping or conflict errors?16:46
*** MrTango has joined #zope16:47
koshis it only specific areas that get slow or any area?16:47
koshalso if you use something like chrome to view a page if you use inspect while on a fast page and then change the url to a slow page the network tab will tell you about each resource being loaded in16:48
koshmaybe something else is stalling the page16:48
koshI have had that happen before where a page got slow for some other reason16:48
koshif a page is slow and you refresh it then is it fast again or still slow?16:48
superdupersheeponce the instance starts to become slow, that's it. i can't restart the instance in the middle of the day, so i just rewrite it off to another zope instance on another machine16:49
superdupersheepi just push it off to one of the 32bit machines, boom, everything's fine16:49
koshif you are load balancing multiple instances why can't you just restart the instance?16:49
superdupersheepi don't see any bad looking entries in my zeo.log16:49
superdupersheepthey're not load-balanced in that way16:50
superdupersheepthere's 4 x machines, each running 8 x instances of Zope16:50
betabugwhat products are ou using on those zopes?16:50
superdupersheepand here's where it gets dirty...16:50
superdupersheepthey are exclusively silva16:50
koshjust as a test for your 64bit systems try setting the cache-size for zeo to 016:50
koshbut leave the regular cache-size outside of it16:50
superdupersheepwhat's your thinking on that one?16:51
koshit will make sure that it is not a local cache issue or cache flipping causing any issues16:51
koshif your local network is fast it should still work fine16:51
koshI know in the case of zeo + zope on the same machine it is faster to make no zeo cache16:51
superdupersheeplocal network is brutally fast as they're actually on blades in the same bladecenter16:52
*** goschtl has quit IRC16:52
koshalso if you could zope 2.13 would probably run vastly faster for you, I get about 3x the performance out of zeo with 2.1316:52
*** goschtl has joined #zope16:52
koshso try just getting rid of the zeo cache on the zope clients16:52
koshit sure seems strange that once it gets slow all the pages get slow with no load average16:53
J1mkosh, zeo caches haven16:53
koshJ1m: what?16:53
J1mkosh, zeo caches haven't "flipped" in a long time.16:53
*** Arfrever has joined #zope16:53
J1mNot since ZODB 3.2.16:53
koshJ1m: he is using an older zope and I did not remember when that was done16:54
J1mprobably zope 2.816:54
superdupersheepzope 2.11 actually16:54
koshI just remember I used to see that but all my current systems have been upgraded to 2.13 which is much faster then previous versions16:55
koshand I almost have all my code ready to convert my sites to blobs16:55
superdupersheepthe reason i suggest memory consumption is an issue is that the slowness often manifests once an instance grows larger than 3G16:55
koshthen I want to test zlibstorage on the stuff remaining16:55
betabuganybody got a script handy to "migrate all objects of a certain meta_type" in a zodb?16:55
J1msuperdupersheep, that's interesting.16:56
J1m3G is a magic number for 32 bits.16:56
superdupersheepobviously on a 32bit machine that isn't possible - it's moving to 64bit that's allowed that to happen16:56
superdupersheepJ1m: exactly.16:56
koshI have had zopes use more memory then that though without any issues16:56
koshhave had them up to 7G without slowdowns16:56
superdupersheepkosh: i'm very pleased for you.16:56
J1mWe use 64 bit machines but out processes are all < 3G because it's too expensive for us to add more than 4G/core.16:57
koshjust that it seems strange that it would slow down at that point16:57
*** zagy has quit IRC16:57
koshI mostly use lots of rackspace cloudservers now and just get more machines16:57
superdupersheepwe're a big deployment, in a UK university16:58
koshbut on a test system I have 8G of ram16:58
J1mgotta go. Good luck.16:58
superdupersheepour Data.fs is 13G, our existing 4 32bit servers have 16GB of RAM each16:58
superdupersheepthey use pretty much all of that ALL the time16:58
J1mBut if you have 8 processes per machine and only 16G, how to you get >3G.16:59
superdupersheepwe don't on the 32bit machines16:59
koshhow many cpus do they have? how many zopes is each one running? how many threads does each zope have?16:59
* J1m goes away now16:59
koshyou could still try setting the zeo cache to 0 anyways on a test and then hit the hell out of it and see if that helps17:00
superdupersheep8 zopes per machine, 8 cores, dunno how many threads offhand17:00
superdupersheepnew machine is set up almost exactly the same, except as i said, for a 64 bit python build17:00
koshwel if you have 8 zopes on a system you would normally run 1-2 zserver-threads on each one17:01
koshif you use the default of 4 that can actually slow things down depending on how you load balance17:02
koshalso it chews up a lot more ram that could be used to make the system faster17:02
superdupersheepwe've got 4 on each, i just checked17:03
superdupersheepthey aren't the problem17:03
superdupersheepthey run perfectly fine - bit slow on occasion but we've got caching to deal with that17:03
koshthey cause memory to be used up much faster but can't realy speed things up since the threads can't truly run concurrently17:03
koshand your load balancers shoudl be distributing the load anyways17:03
superdupersheepi've not seen the 32bit machines exhibit massive slowness like the 64 bit machine does17:03
koshit is very strange17:04
koshespecially if once it gets slow it stays slow17:05
koshthat is why I am suggesting various ways that your normally speed the system up in the first place17:05
mejodoes a simple solution exist to log the access times of zope queries?17:08
koshhowever zope itself works with 64bit since python supported 64bit which is a long time now17:08
superdupersheepseriously, that's not the problem17:08
mejoi mean the time zope (2.10) takes to return the requested page?17:09
superdupersheepthe system is ridiculously fast *when it works*17:09
CIA-107janwijbrand * r121742 megrok.chameleon/ (src/megrok/chameleon/README.txt README.txt): fixup REST syntax and small updates now that do again depend on z3c.pt17:09
CIA-107janwijbrand * r121743 megrok.chameleon/ (CHANGES.txt Preparing release 1.0rc117:09
CIA-107janwijbrand * r121744 /megrok.chameleon/tags/1.0rc1: Tagging 1.0rc117:09
CIA-107janwijbrand * r121745 megrok.chameleon/ (CHANGES.txt Back to development: 1.0rc217:09
superdupersheepthen boom, slowdown, dead17:09
koshmejo: not that I can think of17:09
koshsuperdupersheep: so something has to cause it to suddenly go dead like that and it might not even be zope, maybe silva has some very strange bug17:10
koshon a test system though I would still kill the zeo cache and change the zserver threads to 2 and run a test with that17:11
superdupersheepgod knows. the fact is this kind of crap is almost impossible to debug in any logical way, and you end up playing "change the components one by one until it stops" without any real understanding of what is *actually* happening17:11
koshit is not something I have EVER run into17:13
koshwhen my systems slowed down or stalled out it was specific pages that caused it and once the pages where isolated it was fairly easy to fix17:13
koshusually pages that loaded FAR too many objects or pages that waited for some external connection of some kind17:13
koshlike if a page has something in it for zope to load some remote resource, if that resource takes too long or times out you can stall things out really rapidly17:14
koshhowever those issues I have run into on just about everything and none of them seemed to make debugging those faster17:14
koshthere are some nice tools I have used that will check zope memory usage after every page load and I would just run wget over an entire site with that running and immediately isolate pages that had too many loads17:15
koshhaving the system get slow and stay slow is just very strange to ever have happen17:16
koshespecially at 3GB, that is beyond strange17:16
*** supton has joined #zope17:17
superdupersheeptimes like these i hate being a sysadmin17:17
*** Wu has joined #zope17:20
koshjust recently I actually had to debug postgres locking up17:21
koshno errors in the log17:21
koshbut it would not accept any connections17:21
koshit would run for a few hours and then just stop17:21
kosh  I have also used that and it worked without issues for me17:24
superdupersheepi've looked at that, but to be honest with you, i don't see memory spikes17:25
superdupersheepi see a smooth increase in memory usage to a point (at which point everything is still fine), then at some unspecified time, SLOWNESS AND DEATH17:25
superdupersheepit's stable for a long time before it falls apart17:25
koshon a test I would probably try to crawl the entire site and see if it is a specific url that kills it after that point17:26
koshit just makes no real sense to suddenly get slow at a certain memory usage17:26
*** lcarvalho has joined #zope17:27
*** supton has quit IRC17:28
*** ViicT has joined #zope17:28
lcarvalhoDoes anyone have any documentation about REST in Zope2.12 ?17:28
lcarvalhoActually I am trying to send a PUT method request to Zope server with a PATH_INFO which does not exists.17:29
lcarvalhosuch request is handled by WebDAV.NullResource17:30
lcarvalhoI would like to know, if there is any extension of anything which could be able to provide a REST architecture over WebDAV in Zope2.1217:30
*** slackrunner has joined #zope17:31
lcarvalhoI do not want to publish object for REST methods. I want to handle PATH which does not exists instead of delegating it to WebDAV.17:31
koshno idea about that one17:32
betabughmmm, last I heard, zope did webdav17:36
betabugbut I didn't understand the question, to be honest17:36
koshsee you weirdos later17:50
*** sm has joined #zope17:53
lcarvalhobetabug: I will explain agian17:56
lcarvalhoimagine that you have a zope server running at localhost:808017:57
lcarvalhoso if you sent a PUT request to such server17:57
lcarvalhowith the the PATH_INFO with a resource which does not exists in your Zope instance17:57
lcarvalhoit will be handled by WebDAV17:57
lcarvalhoI would like to be able to add a way to handle such REST requests17:58
lcarvalhoit can be PUT DELETE GET POST basically17:58
*** sm__ has joined #zope18:01
*** sm has quit IRC18:04
*** sm__ is now known as sm18:04
mejobetabug: we didn't find the reason for the freezes yet, but the DeadLockDebugger is a great help nevertheless. We'll keep an eye on requests/queries which put the instance under heavy load and hope to find the reasons soon. Thanks a lot for your help.18:09
betabugmejo: no problem, glad to be of help!18:09
betabugwith the client that I had a similar problem, I seem to have resolved the problem right now :-)18:10
mejogreat to hear!18:12
mejobye and take care18:13
betabugyeah, really glad I could nail that one down18:14
betabughf there18:14
*** __mac__ has joined #zope18:15
*** pjfd2 has joined #zope18:15
*** pjfd4 has quit IRC18:17
*** digitalmortician has quit IRC18:17
*** agroszer has quit IRC18:27
*** mejo has quit IRC18:31
*** sylvain has quit IRC18:32
*** allisterb has joined #zope18:34
*** pjfd4 has joined #zope18:37
*** pjfd2 has quit IRC18:40
*** tiwula has joined #zope19:02
*** fredvd is now known as fredvd|cooking19:07
*** eperez has quit IRC19:14
*** Spanktar has joined #zope19:17
*** mitchell` is now known as mitchell`off19:20
CIA-107yuppie * r121746 zopetoolkit/ (ztk-versions.cfg zopeapp-versions.cfg): - set svn:eol-style19:21
CIA-107yuppie * r121747 zopetoolkit/ (zopeapp.cfg zopeapp-sources.cfg ztk.cfg ztk-sources.cfg): - moved sources to separate files19:21
CIA-107yuppie * r121748 zopetoolkit/ztk-sources.cfg: - added 2 more sources19:21
CIA-107yuppie * r121749 Zope/sources.cfg: - synced sources.cfg with versions.cfg, using the new ztk-sources.cfg19:21
CIA-107yuppie * r121750 CMF/sources.cfg: - removed packages that are now part of Zope/trunk/sources.cfg19:21
CIA-107yuppie * r121751 Zope/sources.cfg: - made it easier to reuse sources.cfg19:21
CIA-107yuppie * r121752 CMF/sources.cfg: - added new zopeapp-sources.cfg19:21
CIA-107yuppie * r121753 CMF/buildout-zope213.cfg: - updated Zope2 versions19:21
CIA-107yuppie 2.2 * r121754 CMF/ (buildout-zope212.cfg buildout-zope213.cfg): - updated Zope2 versions19:21
*** lukasg|4tw has quit IRC19:24
*** hever has quit IRC19:26
*** hever has joined #zope19:31
*** daMaestro has joined #zope19:36
*** TresEquis has joined #zope19:36
*** avoinea has left #zope19:47
*** alexpilz has quit IRC20:21
*** evilbungle has quit IRC20:22
*** TomBlockley has quit IRC20:31
superdupersheepif anyone is still here from earlier, we think we solved our slow zope problem20:32
superdupersheepand it turns out that i'm stupid, and had our ESX VM set up with 1 vCPU instead of 4.20:32
* superdupersheep hangs head in shame, backs slowly out of the room20:33
*** superdupersheep has quit IRC20:33
*** hever has quit IRC20:36
*** Wu has quit IRC20:44
*** davisagli has quit IRC21:01
*** giampaolo has joined #zope21:02
*** davisagli has joined #zope21:03
*** evilbungle has joined #zope21:04
*** tisto has quit IRC21:08
CIA-107jim * r121755 /Sandbox/J1m/customdoctests/ (7 files in 2 dirs): Added spidermonkey-support test and readied for release.21:20
CIA-107jim * r121756 /zc.customdoctests: Support for alternate example languages in doctests21:20
CIA-107jim * r121757 / (zc.customdoctests/trunk Sandbox/J1m/customdoctests): Support for alternate example languages in doctests21:20
CIA-107jim * r121758 /zc.customdoctests/tags/0.1.0: tag21:20
CIA-107jim 0.1.0 * r121759 zc.customdoctests/ *** empty log message ***21:20
CIA-107jim * r121760 zc.customdoctests/src/zc/customdoctests/spidermonkey.txt: Fixed example. Sigh.21:20
CIA-107jim * r121761 zc.customdoctests/src/zc/customdoctests/spidermonkey.txt: doc fix.21:20
CIA-107jim 0.1.0 * r121762 zc.customdoctests/src/zc/customdoctests/spidermonkey.txt: *** empty log message ***21:20
*** __mac__ has quit IRC21:24
*** webmaven has joined #zope21:25
*** lcarvalho has quit IRC21:31
*** zagy has joined #zope21:38
*** sm has quit IRC21:46
*** alecm has quit IRC21:47
*** alecm has joined #zope21:50
*** alecm has joined #zope21:50
*** teix has left #zope22:03
*** zagy1 has joined #zope22:10
*** zagy has quit IRC22:10
*** slackrunner has quit IRC22:46
*** zagy1 has quit IRC22:52
*** ViicT has quit IRC22:56
*** fredvd|cooking has quit IRC23:12
*** __mac__ has joined #zope23:18
*** J1m has quit IRC23:25
*** __mac__ has quit IRC23:27
*** Arfrever has quit IRC23:42
*** davisagli has quit IRC23:50
*** davisagli has joined #zope23:51
*** pjfd4 has quit IRC23:55

Generated by 2.15.1 by Marius Gedminas - find it at!