[Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...

[Dirk Bächle]

Jason,

On 31.03.2015 21:06, Kenny, Jason L wrote:
> [...]
>
> So let me think this out load here. The main issues is that I have properties that I am adding to the node objects and this fails with __slots__ being defined without a __dict__,  because I was adding a cache value to the object, vs having to look up the value on a different object ( ie the attributes object) as this could conflict with who-knows-what that was added with the Tag()/MetaTag() API by the user.
>
> The plus side for having __dict__ added to the base is that it makes it easy to see what is being added to a node and it makes it easy to prototype new stuff in a system. Keep in mind that point of __slots__ is not about removing __dict__, but allowing python to be efficient in pre-allocating memory as an array vs that of independent items as they would be in the __dict__ object. Ideally Cpython should be doing something else here, so __slots__ add no value. ( hmm somebody should fix this...) But until then __slots__ add value for this reason. Add __dict__ by default only mean that the code as any normal python object. One can argue that the restriction we add by using slots in add new member dynamically is a good thing or a bad thing. I just want to point out that the use of __slots__ means that the memory usage is reduced because of block array allocation ( vs lots of small alloc), and the memory layout of the object is more machine friendly ( as any CPU like array for quick acce
 s
s, and hate everything else). This is why we have a space and speed improvement in SCons with the nodes as the node are the prime data structure when we build.
>

I'm mainly interested in reducing memory consumption, and dictionaries require a lot of bytes. So, if you can add your "__dict__" 
without directly assigning a fully-blown dict to any Node instance by default...you win me over in the same second.
But this would require some sort of lazy initialization like I did for the abspath/tpath attributes, and I have the feeling that 
that's not directly what you're after. ;)

> Let me look at what I can do on my end, with a fresh set of eyes.. if you still feel that adding __dict__ is a bad thing. ( will miss the clean property syntax )
>
> There are a few other fixes I have here in parts/overrides/nodes.py that would be nice to have added to the main base. Could you look at these and see if any seem OK for you to add to the Node classes?
>

Definitely, I'll have a look.

Regards,

Dirk