[Scons-dev] SCons Node.FS.Base _get_str method appears to cause a lot of calls to os.stat

[Bill Deegan]

Tom,

Stat's should be cached.. and so shouldn't hit the file system more than
once per file..
@SCons.Memoize.CountMethodCall
    def stat(self):
        try: return self._memo['stat']
        except KeyError: pass
        try: result = self.fs.stat(self.get_abspath())
        except os.error: result = None
        self._memo['stat'] = result
        return result

Also.. It's not the file name which would be changing, just the file name
with path. (since it could be via a repository or variant dir)

-Bill

On Wed, Sep 28, 2016 at 7:48 AM, Tom Tanner (BLOOMBERG/ LONDON) <
ttanner2 at bloomberg.net> wrote:

>
> well, apart from exists uses stat after digging through the code. But the
> point remains. How could the name of the file be changed?1
> From: ttanner2 at bloomberg.net At: 09/28/16 12:40:07
> To: scons-dev at scons.org
> Subject: Re:[Scons-dev] SCons Node.FS.Base _get_str method appears to
> cause a lot of calls to os.stat
>
> This:
>
> 702 -
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> def _get_str
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS.Base-class.html#_get_str>(self):
>
> 703 global Save_Strings
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> 704 if self.duplicate
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> or self.is_derived
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>():
>
> 705 return self.get_path
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
>
> 706 srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> = self.srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
>
> 707 if srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> .stat
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
> is None and self.stat
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
> is not None:
> 708 result
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> = self.get_path
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
>
> 709 else:
> 710 result
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> = srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> .get_path
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>()
>
> 711 if not Save_Strings
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>:
>
> 712 # We're not at the point where we're saving the string
> 713 # representations of FS Nodes (because we haven't finished
> 714 # reading the SConscript files and need to have str() return
> 715 # things relative to them). That also means we can't yet
> 716 # cache values returned (or not returned) by stat(), since
> 717 # Python code in the SConscript files might still create
> 718 # or otherwise affect the on-disk file. So get rid of the
> 719 # values that the underlying stat() method saved.
> 720 try: del self._memo
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>['stat']
>
> 721 except KeyError: pass
> 722 if self is not srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>:
>
> 723 try: del srcnode
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> ._memo
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>['stat']
>
> 724 except KeyError: pass
> 725 return result
> <http://www.scons.org/doc/production/HTML/scons-api/SCons.Node.FS-pysrc.html#>
> 726
>
> appears to be responsible for a huge number of stat calls in our build.
> I'm a little confused as I don't see why the file *name* should change as
> the result of Python code in the SConscript files. Shouldn't it be calling
> self.exists or self.rexists (admittedly I'm not sure which one right now).
>
> _______________________________________________
> Scons-dev mailing listScons-dev at scons.orghttps://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20160928/fe90f1ac/attachment-0001.html>