*** eperez has quit IRC | 00:11 | |
*** tiwula has joined #zope | 00:12 | |
*** avelino has joined #zope | 00:24 | |
jpic | supton: thanks, checking those | 00:28 |
---|---|---|
supton | jpiq: what app framework are you using? | 00:29 |
supton | jpic: ^^^ | 00:29 |
*** J1m has quit IRC | 00:32 | |
jpic | supton: django ... | 00:33 |
jpic | (sorry) | 00:33 |
jpic | i just need nosql for one app and zodb seems like the sanest solution, but it looks like it would be better to use repoze.catalog too | 00:34 |
supton | jpic: so for standalone ZODB, I would strongly recommend using UUIDs for items, have the forward references/relations as attributes of your object storing string (canonical) representation of the UUID, and then index for the reverse/backward relationships using repoze.catalog (which is well documented, if you piece things together yourself). | 00:35 |
supton | jpic: zodb is just a transactional persistent object system, in many respects better than other no-sql choices, but the free-form nature of it leaves indexing choices up to app developers | 00:35 |
supton | jpic: repoze.catalog is the sanest reasonably succinct solution to that problem, with relatively minimal dependencies | 00:36 |
supton | jpic: and decent docs | 00:36 |
supton | jpic: IMHO, you actually may get even better answers (even it slightly OT) in #pyramid | 00:37 |
*** LeoRochael has quit IRC | 00:37 | |
supton | jpic: because there are a decent number of people in the pyramid community who use zodb | 00:37 |
supton | jpic: in place of or in addition to sqlalchemy | 00:37 |
supton | jpic: and the people who use zodb in pyramid don't have the ideological blinders of big-zope-stack like I do ;) | 00:38 |
jpic | supton: thanks a lot for the details, i'll try asking #pyramid too | 00:41 |
supton | jpic: np. quick tip for repoze.catalog: you maintain a DocumentMap between the integer-keyed document ids and the UUIDs… this is totally up to you, optional, but I think a better way to manage this, rather than the more ambiguous integer-ids (esp. if you need some kind of heterogeneous links between stuff you keep in RDMBS via django models and objects in ZODB). Keep in mind your learning curve for doing this in ZODB boils down to: persistent obje | 00:44 |
supton | base classes, transactions, BTrees (different kinds for different containers of things), and something like repoze.catalog on top of that. | 00:44 |
*** LeoRochael has joined #zope | 00:47 | |
*** LeoRochael has quit IRC | 00:56 | |
*** mcdonc has quit IRC | 00:57 | |
*** avelino has quit IRC | 01:02 | |
*** fdrake has joined #zope | 01:04 | |
jpic | supton: thanks ! FTR here's something that was suggested to copy on #pyramid: https://github.com/Pylons/substanced/blob/master/substanced/objectmap/__init__.py | 01:07 |
*** sunew has quit IRC | 01:08 | |
jpic | that looks more suited assuming i get rid of the path stuff | 01:09 |
jpic | if i understand correctly, enabling basic backwards relations between objects consists of 2 moving parts 0) a two way uuid<->object map and 1) a database of relations like an object with (uuid1, uuid2, relationname) | 01:15 |
jpic | supton: does that make sense ? | 01:16 |
jpic | maybe not, maybe just a Persistent with attributes like source_object, dest_object, relation_name and convenience methods would be sufficient .. i should code the use cases i need as tests and find the shortest way to get there ... if you want i'll let you know how it goes - i hope to get this done in a few hours, if not then it means i'm lost bloating it and have to start over ... | 01:19 |
*** mcdonc has joined #zope | 01:22 | |
*** evilbungle has quit IRC | 01:25 | |
supton | jpic: I would not make it that complicated. | 01:31 |
supton | how about this: | 01:31 |
supton | in smaller steps | 01:31 |
supton | store persistent objects, make sure each has a UUID assigned somehow... | 01:31 |
supton | …that somehow could be (a) use plone.uuid library, which uses adapters, or (b) roll-your own convention for storing a UUID as a direct attribute of the object. | 01:32 |
supton | for your case, I suggest (b) as lighter, less framework-committed route to annotate all your objects with a plain-old attribute | 01:32 |
supton | when storing uuids, just store strings. | 01:32 |
supton | e.g. str(uuid.uuid4()) | 01:33 |
supton | which is the canonical rfc 4122 representation of a UUID | 01:33 |
supton | you could choose to store actual uuid.UUID objects, if you like | 01:33 |
supton | your choice | 01:33 |
supton | the uuid.UUID objects take up 4 bytes versus 36 bytes | 01:33 |
supton | so that choice should be based on how many objects you need to store | 01:34 |
jpic | 500K would be a tops, after years and years | 01:34 |
supton | so assuming you have objects that all have UUIDs, you can then store references to the UUID of other objects in some attribute | 01:34 |
* supton employee = Employee() | 01:35 | |
supton | company = Company() | 01:35 |
supton | employee.company = company.uid | 01:35 |
jpic | i've described a use case here: http://dpaste.de/1drPL/ | 01:35 |
supton | so there's your dirt-simple forward ref/relation | 01:35 |
supton | but company knows not of its many employees here | 01:35 |
supton | so you index all employees in a search system like repoze.catalog | 01:36 |
jpic | the test like 29 I think describes what i need to do, but i don't understand why use UUIDs instead of references to objects ? | 01:36 |
supton | and query for employees who have the company's UID | 01:36 |
supton | jpic: you can store references to objects, but this has several pitfalls | 01:36 |
supton | (1) references don't persist across multiple ZODB storages. | 01:37 |
supton | (2) you have to maintain references in two directions (duplicative) | 01:37 |
supton | with an index and identifiers (be they integers that are system-unique like what substanced does IIRC, or uuids), you avoid both problems | 01:38 |
supton | references are exclusively the domain of a directed graph, they point from A-to-B, but gettting the reverse often requires something higher-level | 01:39 |
jpic | yes, it can prove useful to persist across multiple ZODB storages | 01:39 |
supton | you could choose to use references for forward relations AND index UIDs for the reverse, but you end up with the same amount of work and more complexity | 01:40 |
supton | jpic: and migrations are easier with UUID vs. integers, as you never worry about auto-increment collisions | 01:40 |
mcdonc | supton: my suggestion to him was to use https://github.com/Pylons/substanced/blob/master/substanced/objectmap/__init__.py#L402 | 01:41 |
mcdonc | (wholesale) | 01:41 |
mcdonc | that assumes oids are ints, but it's a s/IOBTree/OOBTree away from being able to use uuids | 01:42 |
supton | mcdonc: makes sense | 01:42 |
*** supton_ has joined #zope | 01:45 | |
*** m8 has quit IRC | 01:48 | |
*** supton has quit IRC | 01:48 | |
*** thetet has joined #zope | 01:51 | |
*** davetoo has joined #zope | 01:52 | |
jpic | mcdonc: so every type of relation is a ReferenceSet, which is a two way representation of the relation ? | 01:53 |
jpic | and it depends on ReferenceMap to convert identifiers to actual objects ? | 01:53 |
mcdonc | yes | 01:53 |
*** thetet has quit IRC | 01:53 | |
*** davetoo has left #zope | 01:53 | |
mcdonc | no referencemap doesnt convert identifiers to objects | 01:53 |
mcdonc | that's still your job | 01:54 |
jpic | right, that's the job of ObjectMap ? | 01:54 |
mcdonc | yes | 01:54 |
mcdonc | https://github.com/Pylons/substanced/blob/master/substanced/objectmap/__init__.py#L143 | 01:54 |
mcdonc | but that's a different system | 01:55 |
mcdonc | happens to know about oids, but it's unrelated to references | 01:55 |
jpic | ah, a ReferenceMap is a registry of ReferenceSet ? | 01:56 |
mcdonc | it is | 01:58 |
jpic | thanks, i'm going to try to convert this use case into using that (http://dpaste.de/ZFqyj/) | 02:01 |
jpic | when i do i swear to document it since you guys have put so much effort into sharing with me :D | 02:04 |
jpic | btw if i have one remark to make, i don't understand why i'm import'ing persistent instead of say zodb.persistent | 02:07 |
jpic | i assume that's for BC reasons | 02:07 |
*** River-Rat has joined #zope | 02:09 | |
*** River_Rat has quit IRC | 02:11 | |
*** sp0cksbeard has quit IRC | 02:27 | |
*** Spanktar has joined #zope | 02:29 | |
jpic | if i understand correctly, BTrees.IF.Set serves as a BTree that accepts integers only as keys, if i wanted to use UUID strings, i should replace it with a normal BTree ? | 02:30 |
jpic | no that's not it my bad | 02:33 |
jpic | i should replace IOBTrees by OOBTrees | 02:35 |
*** tiwula has quit IRC | 02:36 | |
jpic | and replace BTrees.IF.set by PersistentMapping ... i feel like i got this almost right | 02:37 |
jpic | not PersistentMapping, but BTrees.family64.OO.Set | 02:39 |
*** mr_jolly has quit IRC | 02:39 | |
*** mr_jolly has joined #zope | 02:46 | |
jpic | yes that seems to work, but i thought that the relations where symetrical, isn't it the case ? here's an example, on line 20 i expected it to work: http://dpaste.de/pOw6N/ | 02:51 |
jpic | is there something that wraps both sourceids and targetids ? | 02:51 |
jpic | it seems that if this is called: themap.connect(book1, author1, refname) then this should also happen: themap.connect(author1, book1, refname) or the relation won't be symetrical | 02:54 |
jpic | ok, it is symetrical, but i don't quite grasp how to predict wether to use sourceids() or targetids(), because i don't think i can predict which side of the relation will be the target and which will be the source | 02:56 |
jpic | mcdonc: why is themap.connect(book1, author1, refname) different from themap.connect(author1, book1, refname) ? it's the same refname | 02:59 |
*** _mup_ has quit IRC | 03:00 | |
*** _mup_ has joined #zope | 03:00 | |
jpic | i might not even need that, so i'm going to try without first - after all you're doing without - so ... it works great ! thanks that looks like exactly what i needed ... My plan is to make an ObjectMap that handles UUIDs, make a repo/package of that and move on, thankssss ! | 03:09 |
jpic | however i got an idea, maybe ReferenceMap could know what object type is source and what is object type is target automatically, that would be cool - of course for relations to self you'd *have* to know if you want source of target | 03:11 |
*** elro has quit IRC | 03:34 | |
*** mr_jolly has quit IRC | 03:37 | |
jpic | FTR as promised, a little docs about what you've taught me: http://blog.yourlabs.org/post/31829697648/how-substanced-handles-relations thanks again ! | 03:37 |
*** Spanktar has quit IRC | 03:48 | |
*** alecm_ has joined #zope | 03:53 | |
*** alecm_ has joined #zope | 03:53 | |
jpic | mcdonc: maybe there's a typo in this docstring, i think it should be "given" rather than "give": https://github.com/Pylons/substanced/blob/master/substanced/objectmap/__init__.py#L194 | 03:53 |
*** alecm has quit IRC | 03:54 | |
*** alecm has joined #zope | 03:55 | |
*** alecm has joined #zope | 03:55 | |
*** alecm_ has quit IRC | 03:57 | |
mcdonc | jpic: thanks! | 04:00 |
mcdonc | fwiw, in referencemap relations are directional | 04:01 |
mcdonc | so connect(foo, bar, reftype) is not the same as connect(bar, foo, reftype) | 04:01 |
mcdonc | because reftype is directional | 04:01 |
jpic | yes, in practice that shouldn't be a problem in my case | 04:08 |
jpic | but you could easily make a SymetricalReferenceSet that knows what class should be source or target | 04:09 |
mcdonc | thats true | 04:09 |
jpic | actually, it looks like you'd set for example book.authors, register that in a ReferenceMap. Then use book.authors, or the ReferenceMap to get books of an author, so you don't need to store symetrical relations at all ? | 04:22 |
*** daMaestro has quit IRC | 04:26 | |
*** J1m has joined #zope | 05:01 | |
*** avelino has joined #zope | 05:04 | |
*** avelino has joined #zope | 05:04 | |
*** tiwula has joined #zope | 05:21 | |
*** River-Rat is now known as RiverRat | 05:24 | |
*** J1m has quit IRC | 05:27 | |
*** kosh has quit IRC | 05:43 | |
*** tiwula has quit IRC | 06:56 | |
*** nande has quit IRC | 07:45 | |
*** dayne has joined #zope | 07:45 | |
*** avelino has quit IRC | 07:49 | |
*** dayne has quit IRC | 07:50 | |
*** KageSenshi has quit IRC | 07:57 | |
*** tisto has joined #zope | 08:23 | |
*** CIA-101 has quit IRC | 08:25 | |
*** CIA-102 has joined #zope | 09:00 | |
*** zagy has joined #zope | 09:11 | |
*** KageSenshi has joined #zope | 09:15 | |
*** yvl has joined #zope | 09:20 | |
*** agroszer has joined #zope | 09:28 | |
mcdonc | jpic: yeah, i use e.g. https://github.com/Pylons/substanced/blob/master/substanced/objectmap/__init__.py#L731 | 09:42 |
*** menesis has joined #zope | 09:42 | |
mcdonc | class Book(Persistent): | 09:42 |
mcdonc | authors = multireference_source_property(BookToAuthors) | 09:43 |
mcdonc | b = Book() | 09:43 |
mcdonc | b.authors = [author1, author2] | 09:43 |
mcdonc | or | 09:43 |
mcdonc | b.authors.connect([author3]) | 09:43 |
mcdonc | etc (see "Multireference" class in there for the API of "b.authors") | 09:44 |
*** maurits has joined #zope | 10:02 | |
*** sunew has joined #zope | 10:06 | |
*** mr_jolly has joined #zope | 10:08 | |
*** alecm has quit IRC | 10:29 | |
*** tisto has quit IRC | 10:30 | |
*** goschtl has joined #zope | 10:32 | |
*** avoinea has joined #zope | 10:37 | |
jpic | neat, thanks | 10:38 |
*** mr_jolly has quit IRC | 10:42 | |
*** mr_jolly has joined #zope | 10:48 | |
*** thetet has joined #zope | 10:49 | |
*** goschtl has quit IRC | 10:51 | |
*** goschtl has joined #zope | 10:52 | |
*** alecm has joined #zope | 11:21 | |
*** alecm has joined #zope | 11:21 | |
*** elro has joined #zope | 11:25 | |
*** J1m has joined #zope | 11:27 | |
*** tisto has joined #zope | 11:34 | |
*** goschtl_ has joined #zope | 11:47 | |
*** goschtl has quit IRC | 11:48 | |
*** eperez has joined #zope | 11:49 | |
*** goschtl has joined #zope | 11:51 | |
*** goschtl_ has quit IRC | 11:51 | |
*** thetet has quit IRC | 11:53 | |
*** tisto has quit IRC | 11:59 | |
*** evilbungle has joined #zope | 12:05 | |
*** J1m has quit IRC | 12:08 | |
*** goschtl has quit IRC | 12:09 | |
*** RichyB has joined #zope | 12:18 | |
*** J1m has joined #zope | 12:25 | |
*** J1m has quit IRC | 12:27 | |
*** tisto has joined #zope | 12:42 | |
*** teix has joined #zope | 12:48 | |
*** mr_jolly has quit IRC | 12:57 | |
*** menesis has quit IRC | 13:01 | |
*** tisto_ has joined #zope | 13:02 | |
*** tisto has quit IRC | 13:05 | |
*** J1m has joined #zope | 13:07 | |
*** agroszer has quit IRC | 13:12 | |
*** mr_jolly has joined #zope | 13:19 | |
*** fdrake has quit IRC | 13:19 | |
*** nande has joined #zope | 13:20 | |
*** agroszer has joined #zope | 13:25 | |
*** goschtl has joined #zope | 13:25 | |
*** menesis has joined #zope | 13:54 | |
*** MrTango has quit IRC | 13:57 | |
*** J1m has quit IRC | 13:59 | |
*** J1m has joined #zope | 14:00 | |
*** mr_jolly has quit IRC | 14:01 | |
*** tisto_ is now known as tisto|lunch | 14:05 | |
*** dayne has joined #zope | 14:12 | |
*** J1m has quit IRC | 14:12 | |
*** mr_jolly has joined #zope | 14:15 | |
*** elro has quit IRC | 14:29 | |
*** J1m has joined #zope | 14:34 | |
*** J1m has quit IRC | 14:39 | |
*** mitchell` is now known as mitchell`off | 14:45 | |
*** zagy1 has joined #zope | 14:47 | |
*** goschtl has quit IRC | 14:48 | |
*** zagy has quit IRC | 14:49 | |
*** fdrake has joined #zope | 14:53 | |
*** River_Rat has joined #zope | 14:54 | |
*** RiverRat has quit IRC | 14:57 | |
*** goschtl has joined #zope | 15:02 | |
*** sunew has quit IRC | 15:11 | |
*** teix has quit IRC | 15:12 | |
*** teix has joined #zope | 15:12 | |
*** kosh has joined #zope | 15:19 | |
*** LeoRochael has joined #zope | 15:35 | |
*** RichyB has quit IRC | 15:46 | |
*** eperez has quit IRC | 15:46 | |
*** Wu has quit IRC | 15:46 | |
*** kosh has quit IRC | 15:47 | |
*** Wu has joined #zope | 15:48 | |
*** eperez has joined #zope | 15:50 | |
*** goschtl has quit IRC | 15:51 | |
*** kosh has joined #zope | 15:51 | |
*** mr_jolly has quit IRC | 15:52 | |
*** elro has joined #zope | 15:56 | |
*** RichyB has joined #zope | 16:00 | |
*** tisto|lunch is now known as tisto | 16:05 | |
*** J1m has joined #zope | 16:06 | |
*** sp0cksbeard has joined #zope | 16:06 | |
*** __mac__ has joined #zope | 16:06 | |
*** mr_jolly has joined #zope | 16:10 | |
*** mitchell`off is now known as mitchell` | 16:18 | |
*** menesis has quit IRC | 16:19 | |
*** benji has joined #zope | 16:20 | |
*** menesis has joined #zope | 16:26 | |
*** mr_jolly has quit IRC | 16:36 | |
*** RichyB has quit IRC | 16:37 | |
*** RichyB has joined #zope | 16:39 | |
*** mr_jolly has joined #zope | 16:41 | |
*** goschtl has joined #zope | 16:51 | |
*** goschtl has quit IRC | 16:59 | |
*** thetet has joined #zope | 17:09 | |
*** eperez has quit IRC | 17:13 | |
*** mr_jolly has quit IRC | 17:29 | |
*** eperez has joined #zope | 17:30 | |
*** dayne has quit IRC | 17:48 | |
*** menesis has quit IRC | 17:56 | |
*** menesis has joined #zope | 17:57 | |
*** mr_jolly has joined #zope | 17:58 | |
*** kosh has quit IRC | 18:07 | |
*** tisto has quit IRC | 18:20 | |
*** tiwula has joined #zope | 18:25 | |
*** zagy1 has quit IRC | 18:31 | |
*** agroszer has quit IRC | 18:41 | |
*** les_sylvains has joined #zope | 18:47 | |
*** __mac__ has quit IRC | 18:59 | |
*** maurits has quit IRC | 19:01 | |
*** benji has quit IRC | 19:03 | |
*** RichyB has quit IRC | 19:03 | |
*** RichyB has joined #zope | 19:03 | |
*** thetet has quit IRC | 19:04 | |
*** thetet has joined #zope | 19:05 | |
*** mr_jolly has quit IRC | 19:23 | |
*** supton has joined #zope | 19:23 | |
*** avoinea has quit IRC | 19:44 | |
*** benji has joined #zope | 19:45 | |
*** mcdonc has quit IRC | 19:48 | |
*** les_sylvains has quit IRC | 19:49 | |
*** River-Rat has joined #zope | 19:52 | |
*** River_Rat has quit IRC | 19:55 | |
*** eperez has quit IRC | 19:57 | |
*** mitchell` is now known as mitchell`off | 19:59 | |
*** Pumukel has joined #zope | 20:04 | |
*** Spanktar has joined #zope | 20:06 | |
*** les_sylvains has joined #zope | 20:08 | |
*** mr_jolly has joined #zope | 20:14 | |
*** mr_jolly has quit IRC | 20:15 | |
*** sm has quit IRC | 20:15 | |
*** thetet has quit IRC | 20:15 | |
*** nande has quit IRC | 20:17 | |
*** zagy has joined #zope | 20:21 | |
*** menesis has quit IRC | 20:22 | |
*** sm has joined #zope | 20:26 | |
*** elro has quit IRC | 20:26 | |
*** evilbungle has quit IRC | 20:36 | |
*** TuomasT has joined #zope | 21:12 | |
TuomasT | I run Zope 2.10.11-final. I am unable to set "Cache-Control: max-age=xxx" tag. It seems zope removes max-age always from cache-control header | 21:16 |
TuomasT | Does this make any sense? | 21:16 |
J1m | no | 21:16 |
TuomasT | For example, if in a python script at root of zope I specify: context.REQUEST.RESPONSE.setHeader('Cache-Control','max-age=300, must-revalidate'), then wget -S shows "Cache-Control: must-revalidate" | 21:17 |
J1m | I'm sure Zope isn't stripping such headers. Some other product you're using may update them. I'm not saying I know what's going on, I just don't think Zope would do that. I'd bet .05$ that it' | 21:19 |
J1m | s some 3rd-party thing. | 21:19 |
TuomasT | J1m: Ok. Will check my Products dir | 21:19 |
*** teix has quit IRC | 21:22 | |
TuomasT | J1m: Ok. The problem is my wget setup and proxy its using. Duh. | 21:22 |
TuomasT | I spent 1-2 hours debugging this | 21:23 |
J1m | You got off cheap then. :) | 21:24 |
*** menesis has joined #zope | 21:24 | |
*** mcdonc has joined #zope | 21:25 | |
J1m | benbangert, did we decide to use a namespace for python-zk-related projects? | 21:44 |
J1m | I'm about ready to (have a student) start working on porting zc.zk to ise kazoo. | 21:45 |
benbangert | J1m: nope | 21:47 |
benbangert | J1m: I think the only thing he needs to add is the service stuff | 21:48 |
benbangert | I have children/data watchers and the rest of the recipes | 21:48 |
*** __mac__ has joined #zope | 21:49 | |
* fdrake is in favor of namespaces; maybe zk.* would be cool. :-) | 21:53 | |
J1m | right, but I want to name the new thing. It won't be zc.zk. | 21:53 |
J1m | right, I was thinking zk | 21:53 |
J1m | so zk.service | 21:54 |
J1m | benbangert, were you suggesting that he add the service stuff to kazoo? | 21:54 |
*** m8 has joined #zope | 21:55 | |
benbangert | no, separate package should be good | 21:55 |
J1m | but no agreed namespace. | 21:56 |
*** elro has joined #zope | 21:56 | |
benbangert | nope | 22:01 |
benbangert | I think zk is fine for a namespace for additional packages | 22:02 |
J1m | k | 22:06 |
J1m | Then I'll start zk.service | 22:06 |
benbangert | cool | 22:08 |
*** River_Rat has joined #zope | 22:16 | |
J1m | actually, I think I'll split the propery-related bits from the service-registration bits. So zk.property. | 22:17 |
*** motto has joined #zope | 22:18 | |
*** River-Rat has quit IRC | 22:18 | |
J1m | and maybe the text-import/export into zk.textrepresentation. | 22:18 |
motto | 8 | 22:19 |
J1m | and the testing bits into zk.testing. | 22:19 |
*** motto has quit IRC | 22:19 | |
*** River_Rat is now known as RiverRat | 22:23 | |
*** River_Rat has joined #zope | 22:28 | |
*** RiverRat has quit IRC | 22:31 | |
*** benji has quit IRC | 22:33 | |
*** zagy has quit IRC | 22:43 | |
*** __mac__ has quit IRC | 22:53 | |
*** River_Rat is now known as RiverRat | 23:04 | |
*** TuomasT has quit IRC | 23:08 | |
*** les_sylvains has quit IRC | 23:35 | |
*** RichyB has quit IRC | 23:42 |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!