[Scons-dev] MSVC 2017+ Toolset Support

[Bill Deegan]

Can I end up with 14.1 installed in both a MSVC2017 and MSVC2019 install
(and of course also all  the flavors of MSVC, build tools, express,
community, enterprise), for a possible total of 8 separate installs? (or
some smaller non singular number?)

I'm not a fan of the more complicated syntax for MSVC_VERSION and would
rather split out the selection to either separate variables for product,
toolset, and product version and/or a callable which handles selection.
MSVC_VERSION -> sounds like after MSVC2017 defaults to latest installed
toolset unless you set toolset version
MSVC_PRODUCT_VERSION -> Enterprise, Professional, Community, Express, or
not set
MSVC_TOOLSET_VERSION  -> This may only apply for MSVC2017 or above (or
could be used if MSVC_VERSION is not set for MSVC < 2017, though that would
likely introduce confusion

MSVC_SELECTOR -> Callable (too allow more flexibility than static versions
above)

Otherwise you end up encoding the information in a string and then you have
to decode it.
Seems more complicated than necessary, and we don't have any other syntaxes
embedded in variables thus far.

I realize you have lots of time invested in your MSVC_VERSION
implementation you've explained, but the goal here is to be consistent with
expected behavior in the face of changed MSVC.

(And yes. I know.. "Foolish consistency is the hobgoblin of little minds",
but I think in this case it's not foolish)

Welcome feedback on above.




On Sun, Aug 9, 2020 at 2:50 PM Joseph Brill <joseph.c.brill at gmail.com>
wrote:

> This email is an attempt to focus the discussion concerning support for
> MSVC toolsets moving forward in a rational, dispassionate discourse.
>
> I am advocating a minor shift in orientation or "world view" for MSVC 2017
> and later: the semantic transition from thinking of the msvc version as a
> product to thinking of the msvc version as a toolset.  This actually makes
> the implementation of toolset support easier.
>
> There is a working version of the support for toolsets at varying levels
> of precision that can be enabled and disabled.  The intent is that a
> command-line switch could be added to enable the toolset specific
> capabilities.  By default, the toolset specific capabilities would be
> disabled in which case the code behaves exactly like the current master.
> This would allow field testing and evaluation of the functionality in other
> environments and to easily test the differences between the current master
> and the toolset functionality.  The functional aspects are feature
> complete.  The working version is a product of more than one man-month of
> full-time development activity and the result of the third iteration of
> implementation strategy.
>
> Existing PR #3717 was closed and the branch renamed due in part to make it
> easier to identify and maintain for myself as I fear that I might be time
> limited in the near term and due in part to the existing discussion
> unraveling fairly quickly with dim prospects of inclusion moving forward.  Despite
> being closed, I recommend viewing the "documentation" (the PR text and the
> first comment) and the vc.py code difference in PR #3717.  At any time the
> PR request can be reopened.  I would need a git guru to walk me through
> associating the new branch name with the "old" PR.
>
> The implementation knocks an item or two off the internal vc.py TODO list
> such as a single vswhere query and processing the json results for all
> known installations.  Ranked toolset lists are constructed by host/target.
> The ranking takes into account the toolset version and product type.  A
> single toolset instance may be a member of multiple toolset lists.
> Expensive lookups are "runtime memoized/cached" so that they are only
> computed once.  While there is a fair amount of initialization, the
> processing burden is based on the number of products installed and the
> number of toolsets for each product.  Typical environments likely have a
> single product installation with a spartan number of different toolsets
> installed. There is nothing particularly "clever" done in the
> implementation and should not be a maintenance burden.  The bulk of the
> toolset support implementation follows the code in the existing master
> (i.e., the bottom of the file).  The modifications to support toolsets
> weighs in at approximately 1400 lines of code.
>
> Prior to MSVS 2017, there was a direct 1:1 mapping between the msvc
> *product* and the msvc *toolset*. The toolset might be upgraded in an
> update or service pack but there was no way to select a specific toolset as
> side-by-side installations were not supported.  That changed with the
> release of MSVC 2017 which allows multiple side-by-side toolset
> installations by target.  MSVC 2017 and later, can be viewed as *toolset* oriented
> (i.e., a MSVC 2019 installation can be viewed as a container for 14.1X,
> 14.2X, and transparently the 14.0 toolsets).  Moving forward, I believe
> that it will be advantageous to change from a *product* orientation to a
> *toolset* orientation.  This change is only for the 2017 and later
> installations and currently really only affects 2017.
>
> With MSVC 2017 and MSVC 2019, an SCons version request of "14.1" or
> "14.2", respectively, *is a toolset request* that cannot be specified at
> a finer granularity. The current msvc behavior uses the *default toolset* which
> is generally the newest installed *toolset* version when the
> "--vcvars_ver" argument is not specified for the vcvars batch file.
>
> For example, a version request of "14.1" is likely to use the
> "14.16.27023" toolset when MSVC 2017 is installed. Effectively, the *product
> request* is a *toolset request* identical to "find the newest toolset
> version installed that starts with '14.1'". However, "14.2" (MSVC 2019)
> will return a *toolset* based on the latest update which likely varies
> across users and computers.  Currently, a request for "14.1" will fail if
> MSVC 2019 is installed with the 14.1 toolset.  With a toolset orientation,
> a 14.1 toolset request would be satisfied first by the native product
> (i.e., MSVC 2017) for the toolset and then by the latest product that
> contains the user specified toolset for a given host/target combination.
> This two-stage query, if necessary, is implemented in the working toolset
> oriented version.
>
> Currently, the actual toolset versions used for 2017 and 2019 are based on
> end-user's installation options and frequency of updates. While two
> different users may be using the same *product* (e.g., 2019) they may not
> be using the same *toolset*. Similarly, the installed toolset version is
> not necessarily the same for all host/target combinations within a given
> product installation.  The current implementation uses a default *toolset* within
> a product across different users and environments, which may not be the
> same for all instances, but currently cannot use a *toolset* across
> product installations.
>
> There may be many reasons (e.g., enterprise and/or customer requirements)
> that build scripts are tied to a specific toolset.  If a build depends on
> the 14.1 *toolset* it hardly seems as important in which *product* container
> it is installed from an end-user's perspective given the two-stage
> selection rule described earlier.  Given the same toolset, the produced
> binaries should be binary compatible independent of the product
> installation.  The same issue will arise when the next MSVC *product* is
> released and end-users desire to use the 14.2X *toolset* combination in
> the new product container.
>
> At present, the shift in toolset orientation would only affect user's that
> desire the 14.1 toolset and do not have MSVC 2017 installed.  In this case,
> the only other product that can be used is MSVC 2019.  This isolates the
> effects of changes to a single version of MSVC.
>
> This could have a tangible side-effect of making build scripts more
> portable: the build scripts can be tied to a toolset rather than a
> toolset/product combination.  Although if it is desired to specify the
> toolset/product combination that is supported as well.
>
> The MSVC_VERSION format was expanded to allow:
>
>    - specifying an individual toolset at varying degrees of precision:
>       - "14.1"
>       - "14.16"
>       - "14.16.27"
>       - "14.16.27023"
>    - specifying individual product types (attached to a specific toolset
>    or a product):
>       - Enterprise: "14.26Ent"
>       - Professional: "14.2Pro"
>       - Community: "14.26.27Com"
>       - Build Tools: "14.2BT"
>       - Express: "14.1Exp"
>    - a right assignment operator ("->") that maps a specific toolset to a
>    defined product version with an optional product type:
>       - "14.0->14.2" use the the 14.0 toolset via a 2019 installation
>       - "14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools
>       installation
>       - "14.1->14.2Com" use the 14.1 toolset via a 2019 Community
>       installation
>
> The translation of the extended version specification to a supplementary
> internal data structure is done in exactly one source code location:
> msvc_setup_env.  All existing code continues to operate on the "NN.M"
> format for MSVC_VERSION as before.  Existing msvc/msvs test suites pass all
> tests.  However, issues with the extended format being used prior to entry
> in vc.py script have yet to be identified, if any.  The working version is
> completely self-contained in the vc.py script.
>
> Without a product qualifier (e.g., "BT"), there is a *subtle but
> significant* change semantically: the product selected will be the
> newest/latest toolset that matches the user specification within installed
> toolsets for a given (host,target) combination. Effectively, without a
> product qualifier, any product type could be returned based on the
> installed toolsets. This only affects multiple installations of the same
> product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.2BT).  For end
> users that only have one product installed, there is no reason to specify a
> product type.
>
> This means that a user does not need to specify "14.1Exp" explicitly if
> there is a single 14.1 product installation. On the flip side, it also
> means that "14.1Exp" may be returned for a "14.1" request. In local
> testing, "14.1Exp" was returned for an "arm" target. In the local
> installations of 14.1 Community and 14.1 Express, the "newest" toolset for
> an arm target was in the 14.1 Express installation as it was installed the
> most recently.  With a product qualifier, a specific product version and
> type is selected.
>
> To be clear, I do not need any fixes, enhancements, and am not reporting
> any bugs.  The toolset version development was done in the spirit of
> contributing to software that I have used since 2.X.  I had a development
> window to contribute to the windows/msvc support and in process address two
> of the outstanding msvc items on the issues list.  I was motivated by
> both the post in the scons users mailing list shown below and issue #3664.
> As I am familiar with the msvc detection code, my initial thoughts were
> "how hard could it be?"
>
> Regards,
>
> Joe
>
> https://pairlist4.pair.net/pipermail/scons-users/2020-June/008216.html
>
> >* The problem is scons don't find any 14.1(2017) installation, he only find
> *>* 14.2(2019).
> *> >* Is there something I could do to tell scons that 141 is installed somewhere
> *>* in vs2019 installation?
> *
> at the moment, no... it's a work in progress.
>
> See https://github.com/SCons/scons/issues/3664
>
> There wasn't really a "Windows native" person involved in the most
> recent times, so we didn't end up understanding about the model of
> multiple versions under one VS install - and in particular that people
> would want to get 14.1 from VS2019 rather than keep VS2017 installed.
> For the most recent VS installs, vswhere is used to detect them, but we
> find only the "default version" under each product vswhere reports, so
> 14.2.latest for 2019, 14.1 for 2017.
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20200810/5b7a801a/attachment-0001.html>