| *** srichter has joined #zodb | 02:55 | |
| *** srichter has quit IRC | 03:02 | |
| *** srichter_ has joined #zodb | 03:04 | |
| *** JaRoel has quit IRC | 09:32 | |
| *** agroszer has joined #zodb | 10:04 | |
| *** JaRoel|4d has joined #zodb | 10:43 | |
| *** JaRoel has joined #zodb | 13:07 | |
| *** JaRoel|4d has quit IRC | 13:13 | |
| *** agroszer has quit IRC | 13:47 | |
| *** JaRoel has quit IRC | 13:49 | |
| *** JaRoel|4d has joined #zodb | 13:49 | |
| *** agroszer has joined #zodb | 13:57 | |
| *** agroszer has quit IRC | 14:29 | |
| *** srichter_ has quit IRC | 15:19 | |
| *** agroszer has joined #zodb | 15:20 | |
| *** fdrake has joined #zodb | 16:04 | |
| *** agroszer has quit IRC | 16:46 | |
| J1m_ | LjL, re indexing, if your needs are simple, it's usually best to use BTrees mapping from your index keys to sets of matching objects. | 16:49 |
|---|---|---|
| J1m_ | If you want something more full featured, see zope.catalog and the many derivative projects. | 16:50 |
| *** J1m_ is now known as J1m | 16:50 | |
| *** nueces has joined #zodb | 17:51 | |
| *** nueces has quit IRC | 18:46 | |
| *** nueces has joined #zodb | 19:02 | |
| *** nueces has quit IRC | 19:03 | |
| *** nueces has joined #zodb | 19:04 | |
| *** JaRoel|4d has quit IRC | 19:24 | |
| *** nueces_ has joined #zodb | 19:47 | |
| *** JaRoel|4d has joined #zodb | 19:49 | |
| *** nueces_ has quit IRC | 19:52 | |
| *** nueces has quit IRC | 19:52 | |
| *** JaRoel|4d has quit IRC | 19:53 | |
| *** nueces has joined #zodb | 19:53 | |
| *** JaRoel|4d has joined #zodb | 19:54 | |
| *** vangheem has joined #zodb | 20:05 | |
| vangheem | J1m: Hi, did you notice: https://bitbucket.org/zc/zc.zrs/issue/2/242-egg-installation-fails-with-import ? I have a pull request that fixes it. | 20:06 |
| J1m | Nope. I'll look at it. Thanks. | 20:09 |
| J1m | Merged. | 20:12 |
| vangheem | thanks | 20:13 |
| vangheem | J1m: a release would be helpful too. no worries if you don't have time though, I'm just poking | 20:14 |
| J1m | Yeah, can't do that right now, but I'll try to get to it soon. | 20:15 |
| vangheem | thanks | 20:15 |
| *** nueces has quit IRC | 20:21 | |
| vangheem | J1m: I'd also like to look into being able to do master-master replication to zrs some day. Is there anything obvious blocking this from being possible or was it never been looked into? | 20:25 |
| J1m | Not sure what you mean by master-master. | 20:26 |
| vangheem | j1m: two separate write zeo servers that sync together | 20:27 |
| *** JaRoel|4d has quit IRC | 20:27 | |
| vangheem | J1m: so multiple primaries for the terminology zrs is using | 20:28 |
| J1m | That's what I wanted to do years ago, but the author of ZRS 1 decided to do primary/secondary. | 20:28 |
| *** nueces has joined #zodb | 20:28 | |
| J1m | A multiple-master scheme, which would be based on voting, would have little (likely nothing) in common with ZRS 2. | 20:29 |
| *** JaRoel|4d has joined #zodb | 20:30 | |
| J1m | It's not clear that it would be a win. The coordination (voting) overhead could swamp the benefits of multiple masters. Still, it might be worth the effort if someone had the time. | 20:30 |
| J1m | Of greater interest in the short to medium term would be some sort fo protocol for automatically electing new masters if a master fails. This is, for example, what ZooKeeper does. | 20:31 |
| vangheem | yes, that'd work too | 20:32 |
| vangheem | however, flipping a secondary over to a primary can be annoying | 20:32 |
| J1m | A much easier performance win, for some definition of easy, would be to rewrite ZEO servers in a better server language. | 20:32 |
| vangheem | because you have to change config, restart | 20:32 |
| J1m | Right, but in practice, this doesn't happen often. | 20:33 |
| vangheem | heh, need to find someone who know erlang and wants to rewrite zeo :) | 20:33 |
| J1m | I'm dating scala atm. | 20:34 |
| J1m | See https://bitbucket.org/zc/zc.zrs/wiki/Project%20Ideas | 20:34 |
| J1m | The first idea is an incremental step that automates the switchover process. | 20:34 |
| J1m | The biggest issue for us in switching is the temporary loss of service. | 20:35 |
| vangheem | yes, this is an easier idea | 20:35 |
| J1m | This means planned failovers tend to be in off hours (fortunately for us, we have those). | 20:36 |
| vangheem | but, as you say, it only hits the planned failover use case | 20:36 |
| J1m | I'd like add an option to ZEO clients to be able to block (for a few seconds) rather than error on disconnection. | 20:37 |
| J1m | An unplanned failover is a traumatic event no matter what you do, because you may have a loss of data. | 20:38 |
| J1m | I'm still not comfortable doing that automatically. | 20:39 |
| vangheem | yah, I suppose | 20:39 |
| J1m | A step toward making that safer would be to record tids somewhere, like ZooKeeper, as they are committed. | 20:40 |
| vangheem | not needing to restart in order to change primary addresses would help a lot to make this easier | 20:40 |
| J1m | Yes, but even on out largest databases, restarts only take a few seconds. Now that indexes are written and read much fater than they once were. | 20:41 |
| J1m | (Although our largest database is on an ssd :) | 20:41 |
| vangheem | right, I guess it's really the applications that are the problem | 20:41 |
| J1m | But you aren't restarting the app. | 20:41 |
| J1m | Only the ZEO server. | 20:42 |
| vangheem | at least in my experience, if I restart a zeo, the client will lose connection | 20:42 |
| vangheem | I never restart a zeo while the client is turned on | 20:42 |
| vangheem | and large plone sites can take a minute to start up... | 20:42 |
| J1m | The client connection code is a disaster. | 20:43 |
| J1m | I can barely get it in my head and can only hold it there a matter of seconds. | 20:43 |
| J1m | Way overcomplicated and brittle. | 20:43 |
| J1m | Having said that, we rarely have a problem with clients reconnecting. | 20:44 |
| J1m | When we do database maintenance, and restart the servers, we'll have >100 clients that reconnect with no problem. | 20:44 |
| J1m | occasionally, one or 2 will fail to reconnect. | 20:45 |
| J1m | Someone recently submitted a patch to fix some problems in the client connection logic. | 20:45 |
| J1m | That logic really wants to be rewritten to be asynchronous, rather than bing threaded, as it is now. | 20:46 |
| vangheem | I'll have to try it again then, maybe I don't need to be as afraid as I am to do it | 20:46 |
| J1m | One thing we found is that the read-only-secondary mechanism isn't very reliable, or for that matter, the logic for recovering from gettinga read-only server when you want to read-write server. | 20:48 |
| J1m | For that reason, we run primaries and secondaries on different ports. | 20:48 |
| *** nueces has quit IRC | 20:48 | |
| J1m | To automate role swaps, we'll need to fix this. | 20:49 |
| J1m | To clarify, the logic in ZEO for handling read-only secondaries when a read-write secondary is needed. | 20:52 |
| J1m | Using secondaries as read-only servers works just fine. | 20:52 |
| vangheem | yes | 20:54 |
| vangheem | so annoying thing… read-only clients throw errors on trying to write. It'd be nice to be able to silence those errors | 20:54 |
| vangheem | right now I just abort every transaction | 20:54 |
| vangheem | some apps try to write on read... | 20:54 |
| J1m | Your approach is interesting if you *know* that missing the writes is harmless. | 20:55 |
| J1m | In general, hiding these sorts of errors seems like a bad idea. Better to fix the apps. | 20:56 |
| vangheem | often it's just things like scales that are generated on demand | 21:01 |
| vangheem | okay to not save to the database since they'll still be cached | 21:02 |
| *** davisagli has quit IRC | 23:17 | |
| *** vangheem has quit IRC | 23:29 | |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!