[Scons-dev] SCons Node.FS.Base _get_str method appears to cause a lot of calls to os.stat

[Tom Tanner (BLOOMBERG/ LONDON)]

but it deletes the cache entry during the 1st pass. See line 711.

So whilst I agree that path could change, it would mean
1) You're adding a repository whilst doing the build (not saying that's wrong, but it strikes me as a trifle unusual)
2) You're creating a derived object with the same name as a source object

Possibly there's another case with variant dirs, but I don't use those so I'm not entirely sure

From: bill at baddogconsulting.com At: 09/28/16 16:39:17
To: Tom Tanner (BLOOMBERG/ LONDON), scons-dev at scons.org
Subject: Re: [Scons-dev] SCons Node.FS.Base _get_str method appears to cause a lot of calls to os.stat

Tom,

Stat's should be cached.. and so shouldn't hit the file system more than once per file..
@SCons.Memoize.CountMethodCall
    def stat(self):
        try: return self._memo['stat']
        except KeyError: pass
        try: result = self.fs.stat(self.get_abspath())
        except os.error: result = None
        self._memo['stat'] = result
        return result

Also.. It's not the file name which would be changing, just the file name with path. (since it could be via a repository or variant dir)

-Bill

On Wed, Sep 28, 2016 at 7:48 AM, Tom Tanner (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net> wrote:


well, apart from exists uses stat after digging through the code. But the point remains. How could the name of the file be changed?1
From: ttanner2 at bloomberg.net At: 09/28/16 12:40:07
To: scons-dev at scons.org
Subject: Re:[Scons-dev] SCons Node.FS.Base _get_str method appears to cause a lot of calls to os.stat

This:
 
 702 -    def _get_str(self): 
 703          global Save_Strings 
 704          if self.duplicate or self.is_derived(): 
 705              return self.get_path() 
 706          srcnode = self.srcnode() 
 707          if srcnode.stat() is None and self.stat() is not None: 
 708              result = self.get_path() 
 709          else: 
 710              result = srcnode.get_path() 
 711          if not Save_Strings: 
 712              # We're not at the point where we're saving the string 
 713              # representations of FS Nodes (because we haven't finished 
 714              # reading the SConscript files and need to have str() return 
 715              # things relative to them).  That also means we can't yet 
 716              # cache values returned (or not returned) by stat(), since 
 717              # Python code in the SConscript files might still create 
 718              # or otherwise affect the on-disk file.  So get rid of the 
 719              # values that the underlying stat() method saved. 
 720              try: del self._memo['stat'] 
 721              except KeyError: pass 
 722              if self is not srcnode: 
 723                  try: del srcnode._memo['stat'] 
 724                  except KeyError: pass 
 725          return result 
 726   

appears to be responsible for a huge number of stat calls in our build. I'm a little confused as I don't see why the file *name* should change as the result of Python code in the SConscript files. Shouldn't it be calling self.exists or self.rexists (admittedly I'm not sure which one right now).        _______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
 


_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20160929/b490bc18/attachment.html>