[Scons-dev] SCons performance investigations

[Jason Kenny]

>>C++ now has Conan supported by JFrog. Without support for this, SCons 
>>C++ will
>>fall behind – I believe CMake and Meson are gaining built in support for this
>>C++ source package repository.

I was talking to the Conan team myself in e-mail. They seem to love SCons over other options, but thought everyone one is moving to Meson and Cmake. I think they would be fine with more SCons support, but will need help as "most" customers are not SCons.

>>I am not entirely sure about a switch to GitHub. Having bookmarks and BitBucket supporting them for pull requests has re-enamoured me of Mercurial.
>>The single biggest issue is "hg pull" doesn't update the working tree unlike "git pull". I have really come to hate "git pull" exactly because of this!

Git fetch is what you want. However I don't get why we keep binding bitbucket to hg and github to git. Parts is using git on bitbucket for example and there are many other great git hosting sites ( like gitlab , etc. I personally would not mind moving to git. I don't think that means moving from bitbucket.

Jason


-----Original Message-----
From: Scons-dev [mailto:scons-dev-bounces at scons.org] On Behalf Of Russel Winder
Sent: Thursday, July 27, 2017 12:39 PM
To: SCons developer list <scons-dev at scons.org>
Subject: Re: [Scons-dev] SCons performance investigations

On Thu, 2017-07-27 at 08:57 -0700, Bill Deegan wrote:
> Russel,
> 
> You're forgetting (I think) that SCons does implement (some) of 
> AutoConf/AutoMake's functionality.  Could it be better/more complete? 
> Of course. Is it usable (and used)? very much so.

From what I can see the end user has to handle pkgconfig more or less manually, i.e. ParseConfig is an unsubtle tool, and may well put people off.
Meson's way of doing this is somewhat better. Also, the config.h file stuff is there but a tad awkward. Again Meson seems to do this better. I spotted https://bitbucket.org/scons/scons/wiki/Usin
gPkgConfig
but code should never be on a wiki, it should always be in a DVCS hence https:
//bitbucket.org/russel/sconspkgconfig

Other tools have a GNU install toolkit so I started https://bitbucket.org/russ el/sconsgnuinstallutils

And the SCons platform determination is not sufficiently fine grained. I started something here: https://bitbucket.org/russel/sconsplatformutils

But no-one else in SCons-land seems interested in these things that come as standard in CMake and Meson. Thus small but sometimes decisive negative points for SCons.

> Here's how I see the next two releases:
> 3.0 - PY2/3 compat, VS 2017, some minor performance improvements.
> Post 3.0 - Migrate to GitHub

I think the move from Python 2 to Python 3 has, for me anyway, been a huge success. Being able to use Python 3.5 constructs in SConstruct and SConscript files is a huge plus.

I am not entirely sure about a switch to GitHub. Having bookmarks and BitBucket supporting them for pull requests has re-enamoured me of Mercurial.
The single biggest issue is "hg pull" doesn't update the working tree unlike "git pull". I have really come to hate "git pull" exactly because of this!

> 3.1 - Focus on performance. I think (per the email/wiki page), there's 
> lots of opportunities to speed up SCons, without losing it's 
> SCons'ness..  (If we swap out sconsign implementations we will need to 
> have the version string be 4.0 as we're breaking compatibility)

I am entirely up for speed improvement, but all my projects are really rather small in comparison to those who comment on project size on the lists, so I have never has this as a frustration.

If the .sconsign.dblite is replaced by something faster then yes I think we are making SCons 4.

> Of course everything depends on how much time/effort the community 
> (and
> myself) are able to provide towards improving SCons.

Ever the same for FOSS projects with no big corporate backers.

For myself I have to prioritise the rewrite of the ACCU conference submission and review system now, so I have to work effectively full time on that. I can always chip in bits on SCons stuff, but nothing even close to full time.
 
> Things I'd really like to see happen in the not too distant future:
> * Revamp platform/toolchain - Right now it's left to the user to 
> implement logic to say, if I have these 3 tools, then use all of them, 
> otherwise use these tools.  Think mingw vs msvc on win32 for example

Not sure what this is saying. Sorry, I am probably just missing something obvious.

I wonder if there is some need to look at the C, C++, Fortran, D integration, especially the linker. I think there are a lot of hacks in there.

> * Add support for components - For example a library can specify what 
> include paths, defines, libraries, compiler flags would be used by 
> programs (or components) which use itself.  Parts provides some of this I beleive.
> I would want this as an available API, but not mandatory to use SCons


> * Handle pulling components from repos.. cmake,gradle, etc can do 
> this, I think it's pretty darn useful especially where you may have a 
> project with many repos and you focus on just one.

Has the "integration" with CVCS repositories as locations of sources been deprecated/removed? Should DVCS integration be included? Go and Cargo both support getting code for compilation from Git, Mercurial and Bazaar (soon to be replaced by Breeze) repositories.

C++ now has Conan supported by JFrog. Without support for this, SCons 
C++ will
fall behind – I believe CMake and Meson are gaining built in support for this
C++ source package repository.

I was trying to create a tool to handle Dub repository packages for D projects, and it looks like the idea has legs. I have had to stop working on it due to having to work on the ACCU Conference stuff. I am sure something could be put together for Bintray packages. Cargo/Rust is arguably the better model to follow that Go.

> * Taskmaster improvements (see the need for speed wiki page, and also 
> comments by Jason about Greg Noel's TaskmasterNG which Greg (who's 
> long since left the project worked on, but never contributed, or 
> shared sources with anyone).

I had assumed this had all gone with Greg when he departed the project.

> Most of these bullets are in the nice to have.
> I think improving performance is in the must have as it's the top 
> complaint I hear about SCons.

I still believe getting the issues out of Tigris and into BitBucket (or GitHub if the project is switched from Mercurial to Git and moved).

I think making use of some of the CI tools such as Travis-CI, Codeship, etc.
would be good for the SCons project. This is as much about publicity and marketing as it is about CI.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder