The Python-on-Windows people are at it again. This time they have deliberately broken os.path.expanduser() in 3.8.

Until 3.7, expanduser() treated a tilde at the front of the path given to it the same as a *ix shell, i.e. ‘~/’ was replaced by the current user’s home directory, and ‘~user/’ with that of user user. More specifically, it used the value of the HOME environment variable, or USERPROFILE if the former did not exist, or HOMEDRIVE\HOMEPATH as a final fallback (on NT 3.51, perhaps?).

Then someone complained about that, Python bug #36264, and for some reason unknown to the world, it was decided not to consider HOME anymore.

This (and I’m speaking out of a deep personal animus) was the most misguided decision since splitting the Windows distribution into 22 MSI packages per architecture.

Why?

Because the supposed rationale for the change is that HOME is not a “special” variable on Windows, i.e. it is not something the system/shell cares more about than any other environment variable.

So what? It was documented that it was used by expanduser(), and that is what matters. Explicitly mentioning some behavior in the documentation, without a note that it is an implementation detail subject to change, is what interface contracts are made of.

The documentation does not make any claims that HOME is used because it is of any particularly exalted rank, it simply says it is used. The relevant paragraph could even be read simply as “the user’s home directory, unless overridden with HOME”.

In a particularly cruel twist, the original complainant even pointed out that their problem was actually caused by a bug in MSYS, but the Python-on-Windows maintainer apparently felt that this localized problem with a single tool merited breaking part of the Python standard library for everyone. (The tool was Git, but still …)

Atlassian have made the extremely regrettable decision to stop supporting Mercurial on Bitbucket, suggesting that everyone migrate their repositories to Git. (They say they “considered” developing an automated migrator, but ultimately they decided to wash their hands of the whole affair and just abandon their customers.)

There does not appear to be a viable alternative hosting solution (by which I mean, none of the dozen or so I looked at are any good), and I’m not going to run my own server; I have better things to do with my time.

Unfortunately, this means that a lot of that time I have the aforementioned better things to do with will be spent converting quite a few Mercurial repos to Git, and learning how to use that crime against sanity and humanity, Git.

I have been a user of the excellent TortoiseHg for many years now, but apparently this is now going to change. I hope the project will keep going.

Thanks for nothing, Atlassian.

Microsoft left some things in the Windows 10 1903 release that crept in during development and should have been removed.

• This is called pseudolocalization:

• Something called the “Multiple Provider Notification Application” creates a directory at C:\Service\Trace Files at each boot.

There is certainly a lot more I have not yet encountered.

Intel’s track record with Windows 10 drivers for their network chips is … well, “spotty” might be the word. For some as yet unexplained reason, after a new “functional update” comes out, it always takes Intel months to get a driver release out that will support ANS (Teaming and VLANs) on that Windows version.

Recently they also have removed the GUI for configuring the more advanced features, such as creating VLAN interfaces (it was called “PROset for Device Manager” or something), so now that has to be done in PowerShell. That wouldn’t be so bad, if there was not also another bug somewhere: As long as any ANS VLAN interfaces exist, the basic “*-NetAdapter” cmdlets do not work anymore; they complain about an “invalid parameter” in some WMI operation. Remove all the VLANs, and it works again as if nothing had happened.

In the early announcement for the latest driver release 24.1, Intel said that “[t]he Team and VLAN configuration issues are expected to be resolved in the next SW Release available in early Q2 (April/May)". At least I assume that refers to 24.1, because that is also the first release to support VLANs on 1903 at all, and the above is part of Intel’s standard response to forum questions about VLANs on 1903.

No such luck (or it was not meant to refer to the WMI issue). Still the exact same error.

This is a problem for those who want to rename their interfaces from “Ethernet 1” etc. because Rename-NetAdapter is also affected.

However, there is a workaround using an older (I think) version of the WMI classes. The more recent root\StandardCIMv2\MSFT_NetAdapter class is used by the *-NetAdapter* cmdlets and transitively breaks them all, but the good old Win32_NetworkAdapter class is still around and still works.

To change the interface alias (=name) of a network connection, you can still use both PowerShell

Get-CimInstance -Namespace root\cimv2 `
                -ClassName Win32_NetworkAdapter `
                -Filter "DeviceID=6" `
    | Set-CimInstance -Property @{NetConnectionID="MIRRORING"}

and WMIC

/interactive:off NIC where DeviceID=6 set NetConnectionID="MIRRORING"

. Without the switch in front there will be a confirmation prompt.

You can get the DeviceID from either

nic get DeviceID,Name,NetConnectionID

or

Get-CimInstance -Namespace root\cimv2 -ClassName Win32_NetworkAdapter `
    | Select-Object DeviceID,Name,NetConnectionID

The output is identical, except that wmic does not underline column headers.

Or you can use netsh, if you don’t care about the deprecation warnings.

netsh interface set interface name="Ethernet 1" newname="MIRRORING"

One item on my C++ wishlist is a syntax for inlining typedefs into the base class list on a class declaration:

class CMyClass
    : public ATL::CComObjectRootEx<ATL::CComSingleThreadModel>
    , public ATL::CComCoClass<CMyClass, &__uuidof(MyClass)>
    , public ATL::IConnectionPointContainerImpl<CMyClass>
    , public ATL::IConnectionPointImpl<CMyClass,
                 &__uuidof(IMyClassEvents)>
    , public ATL::IConnectionPointImpl<CMyClass,
                 &__uuidof(IMyClassOtherEvents)>
    , public IMyClass

The two connection points result in a duplicated name, m_vec. To get the list of listeners to deliver any given event to, contortions like ATL::IConnectionPointImpl<CMyClass, &__uuidof(IMyClassOtherEvents)>::m_vec are necessary. These usually become typedefs:

private:
    typedef ATL::IConnectionPointImpl<CMyClass, &__uuidof(IMyClassEvents)> events_cp;
    typedef ATL::IConnectionPointImpl<CMyClass, &__uuidof(IMyClassOtherEvents)> otherevents_cp;

Then a relatively simple otherevents_cp::m_vec leads to the right container, but it requires regurgitating the entire base class type in the typedef.

Something like this would be really nice to have instead:

class CMyClass
    : public ATL::CComObjectRootEx<ATL::CComSingleThreadModel>
    , public ATL::CComCoClass<CMyClass, &__uuidof(MyClass)>
    , public ATL::IConnectionPointContainerImpl<CMyClass>
    , public ATL::IConnectionPointImpl<CMyClass,
                 &__uuidof(IMyClassEvents)> [private: events_cp]
    , public ATL::IConnectionPointImpl<CMyClass,
                 &__uuidof(IMyClassOtherEvents)> [private: otherevents_cp]
    , public IMyClass

Wouldn’t it?

I know it’s because the class is from waaay back when the SBCS APIs were still actually being used, but …

// Extract a const wchar_t*
//
inline _bstr_t::operator const wchar_t*() const throw()
{
    return (m_Data != NULL) ? m_Data->GetWString() : NULL;
}

// Extract a const char_t*
//
inline _bstr_t::operator const char*() const
{
    return (m_Data != NULL) ? m_Data->GetString() : NULL;
}

BSTR is a Unicode string, and the second extractor above converts it to ANSI. This, unfortunately, results in ambiguous-overload errors whenever a _bstr_t is passed to anything that can accept both forms.

There should really be a way to, say, #define _BSTR_T_NO_ANSI_SUPPORT.

Huh? I thought type libraries could only represent the Automation compatible types?

Good to know. #import is much easier than writing all the wrappers myself, particularly because I’ve been rather generous with the property attributes this time.

Problem

You are developing a COM object that supports outgoing events (source interfaces). You get a compiler error similar to

C2440: ‘static_cast’ cannot convert from ‘CMyClass::_atl_conn_classtype *’ to ‘ATL::_ICPLocator<IID_IMyClassEvents> *'. Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast.

on a CONNECTION_POINT_ENTRY() line:

class CMyClass
   : public ATL::CComObjectRootEx<ATL::CComSingleThreadModel>
   , public ATL::CComCoClass<CMyClass, &CLSID_MyClass>
   , public ATL::IConnectionPointContainerImpl<CMyClass>
   , public ATL::IConnectionPointImpl<CMyClass, &__uuidof(IMyClassEvents)>
   , public IMyClass
{
public:
   CMyClass();

   DECLARE_PROTECT_FINAL_CONSTRUCT()

   BEGIN_COM_MAP(CMyClass)
      COM_INTERFACE_ENTRY(IMyClass)
      COM_INTERFACE_ENTRY(IConnectionPointContainer)
   END_COM_MAP()

   BEGIN_CONNECTION_POINT_MAP(CMyClass)
      CONNECTION_POINT_ENTRY(IID_IMyClassEvents)    // <-- C2440
   END_CONNECTION_POINT_MAP()
...

Analysis

The IIDs in the IConnectionPointImpl base class and the CONNECTION_POINT_ENTRY are not the same symbol. The cast fails because it attempts to convert to a base-class pointer that is not a base class, but a separate specialization of the same template.

Solution

Use either the predefined constants (IID_IMyClassEvents) or the MIDL metadata based constants (__uuidof(IMyClassEvents)) in both places; do not mix them.

burn.exe Considered Harmful, or, Why burn Should Burn

The WiX Toolset is a suite of applications for creating and working with Windows Installer packages (.msi files). Today, it is the most elegant way of creating installation packages for Windows, because it provides reproducibility, a logical structure, and, because it uses XML files instead of a GUI, the package definition can be trivially version controlled. All around, it is the optimal solution to a difficult problem.

Windows Installer has some support for installation UI, mostly to determine the installation directory and to select which optional features to include, but other things are possible. It is not the prettiest of UI libraries, neither in visual appearance nor in design technique, but it works, and, after all, an installation package’s UI does not have to be pretty as long as it works.

Recently, support was added to WiX for “bundles” with “bootstrapper applications”. Bundles are collections of MSI packages that are installed in a specific sequence. Conditions can be defined to decide whether any given package is installed at all. The final distributed output is an .exe file. This is a problem.

First, some background

The MSI file format is designed to make software package management, if not simple, then at least tractable. This situation is particular to Windows, because on other platforms with an integrated package manager (yum, apt, pkg, etc.), there is typically only one source of packages (the distribution’s own repository), and it is well understood that third-party packages do not usually work correctly. On Windows, with hundreds of thousands, if not millions, of developers producing their own packages, this model is clearly not viable. Microsoft’s answer was the MSI file format, because a well-designed MSI has a good chance of “being a good citizen” on the system it is installed on, and, possibly more important, the quality of a package can be assessed before it is installed (e.g. by running the ICEs against it or simply by looking through the tables for obvious issues).

An additional complication that mostly affects Windows, though it exists in some manner on all operating systems, is the Registry database. Installing a package usually both places files on the system and adds certain entries to the Registry, such as file type associations and non-user-specific application configuration data.

The price that the designer of an MSI package has to pay is a certain amount of restrictions. There are rules that a correctly designed package has to obey. Some of them are only checked when the package is validated, some are inherent in the structure of the MSI database and cannot be circumvented.

Windows Installer defines a set of “actions”, which are discrete steps in installing a package. Each action acts upon the system, using information from a set of database tables in the MSI file, to transform the system from one known state to another.

If you are not yet familiar with MSI, it is worthwhile to read the previous paragraph again. MSI, alone among all distribution package formats as far as I know, does not follow an imperative approach that describes how the target system should be changed by the installation, but instead uses a declarative model that describes how the system should be when the installation is finished. The knowledge of how to achieve the desired system state, which may involve, e.g., interdependent modifications that have to happen in a specific order, is provided by Windows Installer itself. The package author does not need to be overly concerned about it.

Enter the “custom action”. “Custom” actions are, as the name suggests, provided by the package author. They can basically do whatever they want, including things that the MSI developers deliberately did not make it possible to do.

Immediately after the first version of Windows Installer was released and Microsoft documented the MSI file format, various companies that had made a good living producing software for producing installation packages (Wise, InstallShield, etc.) started offering versions of their software that no longer created opaque SETUP.EXE files, but MSI files. Unfortunately, they either did not understand the reasons for MSI’s strict design or bowed to user demand; they introduced their own custom actions that, in a nutshell, turned the MSI model inside out by using Windows Installer only as an entrypoint to a “legacy” installation package written in their, by now obsolete, installation scripting language. InstallShield is a very good (or very bad) example of this. For several years, users lived in fear of seeing the words “InstallScript” anywhere, particularly in an installation error message. (InstallScript was a separate component from the actual installation package and had to be installed already, in the correct version, etc.; it was “DLL Hell” with setup components.)

There are some operations involved in installing certain kinds of packages that cannot be represented simply as state changes (like adding a file or a registry key can) but need to interact with other software already on the system, such as registering a plugin for something where the “something” only offers a registration helper application that must be called. For such things, custom actions are, regrettably, necessary. However, if something can be done without a custom action, it should be, no matter how convenient the custom action-based approach may appear in comparison.

Back to WiX

The central advantage of the MSI package format is that it is, essentially, data. It does not contain executable code, other than where the package author decides to include custom actions. By inspecting the Features table, you can learn what parts of the package are optional. The Directory table tells you if the package wants to install files anywhere outside its installation base (such as the Windows directory). In the Property table you find all the package metadata, such as version numbers. You can even adapt a package to your particular requirements by designing a transform (.MST file) that will be applied during installation but leave the original MSI file entirely untouched, giving you the opportunity – unheard of before the introduction of MSI – to actually change the result of the installation in ways not necessarily foreseen by the package author.

Most of all, the nature of the MSI package enables seamless deployment. You don’t have to dig up long-forgotten lists of command-line arguments supported by the setup.exe package of the day, you do not even need the ability to run an executable as part of your deployment at all. You simply tell your client management solution (or even a simple group policy object) to install a package, and that package is installed, perhaps with a transform or two.

burn’s bundles clearly were intended to do away with all that, and they succeeded fully, from start to finish. There is no seamless deployment anymore, because now you have to work with command line options again. There is no trivial transformation anymore, because that requires that the bootstrapper application pass your arguments (including the transform file name) to the actual MSI package it is intended for (and none other). It dares the package developer to design UI for prettiness instead of functionality by providing several predefined “themes” and support for custom ones. Finally, it abuses a feature intended for hiding required system components to prevent its bundled MSIs from showing up in the Control Panel’s applications list.

The WiX installer bundle itself is a perfect example of how not to do installation UI: It is entirely nonstandard, moves and blinks to maximally confuse the user, and is quite ugly on top of that. In an environment that does not support deployment of .exe packages (the vast majority of domains that only have group policy available), it is also essentially nondeployable. It does not even provide a download size advantage even though some parts of the bundle are optional and might be downloaded only if required, because the WiX project always ships the full set in a bundle of ~24 MiB (including some redundant files that would only appear once in an integrated package, but they apparently do not make much of a difference in size).

Conclusion

There is very little that can be done with bundles that a well-designed single MSI package cannot do, and what little there is, no installation package ever should do.

Do not use burn. It burns.

I just updated the signature on a number of Excel .xlam add-ins after the signing certificate expired. When I opened each, Excel behaved in three different ways:

• No complaint at all
• Yellow bar, “macros are disabled”
• Scary warning dialog, “problem with this file, signature expired”

All for the same kind of file, signed with the same since-expired cert.

Huh?