Wix way should you go?

Now that I know Rob still reads my mostly stagnant blog, I guess it is the appropriate time to write a long-overdue post.

When I started working with Windows Installer technology, back when it was first introduced with Office 2000, I played around with customizing the package for the IT department to push out a customized version. The tools were quite primitive, the technology was new and largely unknown, and the concept of having blogs, yet alone Microsoft folks blogging, seemed completely foreign. Support was pretty much nonexistent, and much of the documentation was unintelligible. Fast-forward 10 years, and what a difference that makes! Today there are several free and low-cost repackaging tools for transitioning non-Windows Installer based setups to the MSI format, authoring tools, and lots of community support.

Most setup authoring tools have significant issues. Non-MSI or script based installations have issues because they encourage hacks - I can't tell you how many installations I encountered that install services by writing keys to the CurrentControlSet hive and forcing you to reboot merely so the Service Control Manager can pick up that addition. Furthermore, if you are targeting any sort of enterprise where more than one of your setups will be installed IT departments want MSI deployments for very good reasons. GUI based Windows Installer tools fail to do a good job of grouping related things into the same component, and dynamically adding a directory of files at build time breaks patching semantics horribly. Another big disadvantage to these tools lie in the setup author because he or she does not need to understand the underlying technology and can get away with "programming by coincidence" (as described in The Pragmatic Programmer).

I remember several paradigm shifts throughout my experiences with setup technology - nested MSIs, merge module distribution, and chaining installations. During this time the stock price of Rolaids likely skyrocketed. The biggest challenge was attempting to get developers to take a more proactive approach to deployment considerations as they were writing their code. One approach that I took was the use of merge modules - developers of feature-units would package their build output in an MSM that was consumed when building the final product. Using Visual Studio 2005+ with their deployment projects was not only difficult, but downright impossible because of how limited, shortsighted, and buggy deployment projects are. Adding custom actions to these modules involved a complex and convoluted post-build scripting process that nobody understood, but it DID move teams towards the direction of thinking of deployment while coding.

These days, the tag-team of MSBuild plus Wix 3.0 is THE enabler to accomplishing those goals and largely eliminating the disadvantages of the GUI-based tools. Since there is close to a one-to-one correlation of XML elements to the Windows Installer tables, it is quite simple to follow if you understand the underlying Windows Installer engine. To use WiX to author a complete installation, you MUST have an understanding of the Windows Installer engine. To make a few tweaks or additions once the basic skeleton of the installer is laid out, just about any developer can do it provided access to the WiX documentation. I have team members that are NOT setup developers add services, event log sources, and more with no official training.

Some of the more compelling points in favor of WiX is how you can use it to easily and properly make multiple product editions which share components, separate units of related components into their own WXS file(s) for easier understanding and maintenance, and integrate it easily as a first-class citizen into an MSBuild project. No other product is available to my knowledge that accomplishes those goals. Best of all - WiX is free, fast, and easily installable onto any developer machine.

If you are looking to switch authoring tools, take WiX for a test run by using the dark.exe decompiler to convert your existing MSIs and play around with it a bit. Subscribe to the WiX mailing list and ask a few questions. You just might like it.

Congratulations to Rob and the entire team and individuals who have contributed to it, as well as the community of developers who support it via the mailing list on a daily basis. If you are ever in Pittsburgh, let me know. I'll buy you a beer.

Visual Studio 2008 GenerateBootstrapper task and UAC

I am extremely disappointed in the Visual Studio 2008 bootstrapper to say the least. While the concept of a bootstrapper is great, it is obvious that the designers did not look at real-world deployment scenarios.

Let me give one example scenario: Installing prerequisite packages and interacting with Vista's User Access Control (UAC). Most of the time, the author of the bootstrapper (by the way that packages are authored and selected) is pretty confident as to how things are required (and I use the word required here very specifically) to be installed: per-user or per-machine. In some cases, such as the .NET framework and many third party redistributables, this is a per-machine only situation. Perhaps some components could be installed per-user, but generally if one part is installed per-user, then all parts capable of being installed per-user should be.

The above understanding should certainly be incorporated as part of a bootstrapper from the start. If some components require a per-machine installation, UAC should be prompted for once, start the elevated process, and this elevated process could be 'commanded' by the non-elevated bootstrapper main process. If per-user components exist, the option to install all of them could be given to the user, and appropriate ALLUSERS=x or command line arguments could be passed to the chained installations capable of doing per-user installations, and the elevated or non-elevated boostrapper process is used based on the requirements of the package and user selection.

Instead, the VS 2008 bootstrapper prompts for elevation of each chained installation, even though the entire POINT of a bootstrapper is a single entry point and interaction for the user to get a piece of software installed onto a users machine. We as developers consider setups a massively coordinated symphony of prerequisites and third party components. The user considers software they purchase or wish to install as a single piece and a single process, and it should seem this way to them.

For setups that are rather long and/or chained-installation heavy, prompting after each chained installation is aggravating to the user and is completely unnecessary from their perspective. This design flaw requires the user to babysit the entire process. We as developers hope that the user chooses to elevate each one - if not the entire setup fails with a rather hopeless error message requiring a support call.

Corporate users are the only ones who generally care about per-user installations, and generally can follow a dependency chain of installations to push apps to the users. Most of them will NOT use our generated bootstrapper, anyway.

The point of this post is to tell you how to work around the 'elevate for each' behavior of the generated setup.exe by hacking the setup bootstrapper used by the MSBuild GenerateBootstrapper task so that it prompts for elevation once at startup and installs everything elevated. This is not ideal because sometimes it is not necessary to elevate, but it certainly makes the end-user result better for the 'happy path' for 99% of all users.

Normal caveats apply - if you break something, its not my fault. Keep a backup of all modified files. The solution presented here will change the behavior for ALL GenerateBootstrapper build tasks unless as part of your msbuild script you replace the appropriate file with the version that gives you the appropriate results.

The Windows SDK actually contains this bootstrapper, and it can be located here on a default installation on a 'normal' PC: C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Engine\setup.bin. Copy this somewhere and rename it to setup.exe. In Visual Studio 2008, do a File->Open->File and find your setup.exe. This will open up a resource viewer/editor. The RT_MANIFEST 'folder' is what you want to expand. Right-click on "1" and export it, saving it as a text file. Open the text file and find the requestedExecutionLevel node - change the level attribute's value to 'requireAdministrator'. Use CTRL-A and CTRL-C to copy the contents of this file to the clipboard. Back at the resource editor, double-click the '1' entry to get it in the hex editor. Select all but the first three bytes and paste your clipboard contents in there. Save and exit. Now rename the setup.exe back to setup.bin and replace the original file, saving a backup, of course. The next time your build happens, the setup.exe will now use the new manifest, giving you one prompt and doing EVERYTHING elevated.

I won't get into the other issues with this bootstrapper - no autogeneration of MSI logs, no timestamps in its own log, immediate run and exit that hinders seld-extracting archive packages for single-file download, no MSI caching for patch scenarios like the Office team uses, spanning over multiple CDs/DVDs etc. I will wait for Wix 3.5's Burn to hopefully resolve these issues and the ones not mentioned. Rob Mensching, are you listening? I'll even volunteer to help write it.

Trouble ahead for time computations?

I was just reading an article about the new U.S. Energy Bill that intends to extend Daylight Savings Time an additional month (starting in 2007). I wonder what the impact of this is on the Computer Science front. There has not been any chatter on my normally read blogs, but perhaps there will be soon. I have no facts to back up my theory here, but lets play along with my conspiracy theory and possibly learn something about static and dynamic linking of DLLs.

Think about this from the Microsoft C/C++ developer perspective: the date functions are implemented in the C standard library, MSVCRT.DLL (or its post VC6 brethren MSVC*.dll). Lets assume (although I am guessing) that the "rules" for the current daylight savings time are implemented in this same DLL (NOTE: Please see comments). One of these functions is called mktime(). Let me quote from MSDN:

When specifying a tm structure time, set the tm_isdst field to 0 to indicate that standard time is in effect, or to a value greater than 0 to indicate that daylight savings time is in effect, or to a value less than zero to have the C run-time library code compute whether standard time or daylight savings time is in effect. (The C run-time library assumes the United States's rules for implementing the calculation of Daylight Saving Time).

So if we developers assumed that the runtime would take care of conversions for us, and we statically linked to the library, we may have to recompile against a newer link library (with the revised code) for our applications to continue working if this change actually happens. On the other hand, had the developer dynamically linked to this DLL and not copied the pre-dst-updated MSVCRT.DLL to the application folder or application current directory (Remember that in Windows Server 2003 and possibly Windows XP the rules have changed), the DLL can be updated once in the system32 directory (perhaps through Windows Update) and our preexisting applications will be none the wiser and work just fine with the change.

This is more food for thought on the debate of static vs. dynamic linking that could help sway some developers into a different line of thought - I prefer dynamic linking, but in some cases for some applications I do not practice what I preach. For the installation folks, please read the previous link (and the comments - good stuff there on merge modules) - this is why you need to be sure that you are following proper version replacement and marking shared DLL's as shared - otherwise you can cause grief for other applications or your own.

[UPDATE: 10/4/2005 - fix formatting and errant character]

Installing an Internet Explorer ActiveX Control Part II

In part I of this article, I expressed concern over the various methods of installing an ActiveX control normally distributed and/or updated in a CAB file via Internet Explorer. This follow-up post addresses some concerns of a commenter to that last post, and describes what my solution was.

The comment made by Troy was from the perspective of a repackaging solution. Troy is quite correct - using pre and post installation snapshots will grab the changes to the ModuleUsage subkey. However, this solution fails to achieve the other goals for this particular installation. We still have the issue of a later update coupled with the MSI repair (or self-repair) operation. A point I failed to mention but Troy unintentionally reminded me of is compatibility across versions of IE. With the type of software this is, it may have a shelf life that would include IE7. Are the changes made to the system under IE6 applicable to all versions of IE under all OS's? I don't know. Using the API to install the control alleviates most (but not all) of the potential future incompatibility concerns.

I especially like how he disliked option #3 - changing the IE security settings to assure a successful installation, calling the API, and resetting them. I laughed as I wrote this option, as I could not imagine anyone doing this. In the middle of last week, I came across the "Get In the Zone" section of an issue of "Web Team Talking." They actually suggest and provide code for doing just that - although in a different context for a slightly different reason. I happen to agree with the KISS directive - "Keep it Simple, Stupid" - but sometimes to assure a successful operation you need to handle the corner cases.

The last part of his comment - "[this is] why many enterprise customers choose to repackage installations from ISVs" is a completely different blog post. I cannot tell you how true this statement is - and it is usually caused by setup developers, either intentionally or unintentionally. I'll add this one to the list of future topics for sure.

For the sake of discussion we were using the Crystal Reports Web Viewer which can generically be replaced with any properly signed cab file that works in the normal deployment mechanism of IE using the OBJECT tag. I have not tried this with unsigned controls, but I tested the approach using Macromedia's Flash player. You can play along by visiting the Macromedia's Flash download page and getting the CLSID from the OBJECT tag in the page's source. I then downloaded the swflash.cab file from the codebase attribute of the object tag and stored it on my local disk. The example below is going to use this flash cab file and the flash Class ID. Additionally, I am going to assume the reader is familiar with COM, C++, and MFC. The code is a bit lengthy to post here, but using the steps below a reasonable Windows developer should be able to complete this exercise in an hour or two.

I chose MFC since we need to implement IBindStatusCallback. This is easily accomplished by deriving a class from CCmdTarget. Make sure the implementations of QueryInterface(), AddRef(), and Release() call the External...() base class implementations. GetPriority() and OnLowResource() can simply return E_NOTIMPL. All the other methods can return S_OK. Since this is an async callback we need to have an event handle member variable, initialized in the Constructor, and have the event set in the OnObjectAvailable() method. That's the essential plumbing for the callback class. The callbacks for download progress can be implemented for the progress bars in the MSI.

Before we continue, let's make sure we have all the data we need to install the control.

• The class id of the control is {D27CDB6E-AE6D-11cf-96B8-444553540000}. It needs to be in this format as a string (with the braces) so we can call CLSIDFromString to convert it to a CLSID type.
• We are going to use major and minor versions of 0xFFFFFFFF to assure it is always installed.
• Finally, we need a codebase - or place to "download" the cab file from. Since we are installing the cab file locally, the CA needs to be appropriately place in the sequence and we need to get the local path to the file. The path needs to be in appropriate format - for a local disk where the file is at C:\swflash.cab, we need to make it file://c:\\swflash.cab

The entry point code needs to hook up the plumbing - instantiate the IBindStatusCallback implementation class, call ExternalQueryInterface() on it to get the interface pointer. Then we need to create and register the binding context by calling CreateBindCtx() and RegisterBindStatusCallback().

The meat of what we are doing is a call to CoGetClassObjectFromURL() with the parameters described above. The two common returns from this call will be
MK_S_ASYNCHRONOUS, S_OK, or an error. If S_OK is returned, the object is already installed. If the async value is returned, we need to loop until the event in the callback class is set or some predefined timeout elapses. After the event is set, check to see that the object was created based on what was set in the OnObjectAvailable() method.

Based on testing, this method appears to work all of the time, regardless of the various IE security zone settings. This is likely due to using the file:// protocol. Remember that we are dealing with a signed control - with unsigned controls, your mileage may vary. This method was not tested in IE with Server 2003 in Enhanced Security Configuration, so all bets are off there as well.

If you wish to have the installation always grab the "latest" version of the code from a URL, you will need to implement the code to change and restore the IE security settings from the article linked to above and also implement the download callbacks to move the MSI progress bar. Be sure to alert the user that you will be temporarily altering the IE security settings, and make sure when handling a cancel button press from the MSI that you restore the settings. This particular corollary to the problem can be useful if you want to refresh the clients using a simple repair/maintainence/reinstall option after the server-side control was updated without having Bob the network admin do much extra work (or having to release client side patches).

Installing an Internet Explorer ActiveX Control

This post is different from my usual posts, mainly because I have no idea what I am talking about! Well, at least I am admitting it this time!

I am tasked with creating an installation for an intranet type application. I need to create a shortcut to a URL and install a client side ActiveX control. For the sake of argument lets make it a fairly common one, such as the Crystal ActiveX viewer control. Sounds pretty simple, eh?

Let's go over a typical usage scenario. Hopefully, when designing applications and their deployment packaging you have some made-up profiles of typical users. Lets make up a few for our purposes. "Bob the system admin," "Mary the power user," and "Joe the restricted user" should be good enough. We can be a bit more specific and cover XP SP2 users, users with severely restricted ActiveX Browser settings, and non-IE browser users as well, but lets ignore these for now.

Essentially, Joe has a problem. We can give him the URL to the web application or even create a shortcut and/or favorite for him. The problem is, the first time he tries to run a Crystal Report, he gets a security warning, and the Crystal ActiveX control does not install. Of course, it is because he has no permissions to install software, even if everything else we ignored above is in good shape. Incidentally, if we can't figure out that Joe's problem is that he is a limited user, you can try the CDLLogViewer application to find out why.

Bob and Mary never have issues with the system. In fact, if Bob and/or Mary perform the same steps as Joe everything works, and Joe logs in later on, even Joe can view and print reports. The problem is that Bob and Mary would never run reports on the system. Heck, Bob doesn't even launch the application. Bob may even deploy the install package using SMS or Group Policy.

In the corporate network world, pretty much everyone is a Joe. There are a few Privileged Mary's, and even less Bob's. To make all this work, Bob needs to install applications for the likes of Joe. This behavior is not only acceptable, but practically expected.

So how would Steve, our installation guru, make all this work? He would not only need it to work, but be practically bulletproof.

Option 1: The easiest approach. 99% of the time, this is synonymous with the wrong approach. We can extract the Crystal Reports ActiveX viewer cab file, read the .inf file to figure out what it is doing, add the files to a MSI project selecting the proper registration options, create the shortcut file, and be done with the project in under 20 minutes.

Why it doesn't work: Since Joe is not the only user of the machine (remember Mary and Bob?), any updates to the control will cause the MSI-based installation to screw up at some point. If the control is updated by Mary, and Bob uninstalls the application, the component/reference count then causes havoc with Mary by possibly removing the file on uninstall. Effectively, we have precluded the normal installation/updating/removal of the ActiveX Viewer by repackaging it in an MSI. More information on the hazards of this approach can be found on MSDN in the "Registry Details" page. Bad stuff.

Option 2: Install the shortcuts as in Option 1. Call the appropriate API's to install the CAB file from the web server. In this case, (after a bit of digging through MSDN I see this process is called "Internet Component Download" or ICD for short) we need to call CoGetClassObjectFromURL.

Why it doesn't work: Referring to the ICD link above, we assume the web server is up and running at installation time - #1 in the "Download and install process". We also assume that WinVerifyTrust returns a value that is compatible with the setting for ActiveX controls for the Zone in which the server we are pulling the cab file from resides - #2 in the "Download and install process". Last, but not least, we are assuming that good old Bob didn't muck with the CODEBASE setting of the Internet Search Path.

Option 3: Completely decimate the user's machine by querying the IE Security Settings, saving them, adding the server to a trusted zone, modifying the zone parameters to set the ActiveX settings appropriately, call CoGetClassObjectFromURL, after it succeeds, set everything back to the way it was.

This seems like a heck of a lot of work, but it appears to be the only way to bulletproof an installation of an ActiveX component. Plus, we ignored several issues. If the default browser was (or will be) Firefox or even Lynx, a shortcut to this URL will always fail to display the reports. To solve this problem, an EXE wrapper would need to be written to host the IE control to assure that IE is actually the browser engine used to access the page. Joel, the API war is far from over - it just made things more complicated!

Of course, I'll get to play around with these API's and see if installing the cab file from a file:// UNC path changes anything. I'll report back what I find. If anyone else has already "solved" this problem using an approach I haven't considered, please let me know. In all likelihood, I'll end up calling CoGetClassObjectFromURL, and if it fails, try to provide some meaningful feedback to the user as to why the installation can not succeed and let Bob worry about the corner cases. After all, he ultimately created them.

UPDATE: Please see Part II of this post!

More Setup Pet Peeves

Everywhere you look there are more setup pet peeves that keep popping up. Many of these new ones are in my top 20 list - some of them aren't.

Microsoft's JeffDav writes one about two installation issues - a 3rd party application and a Microsoft application. The second one was probably a MSI based installation, and I blame the MSI engine's design on this one - what ever happened to "Couldn't copy file from source media, retry" dialogs?

Raymond Chen chimed in with his own story, and the usual 2 billion responses to it - some of the catchy ones captured below.

Janus points out that several Java based apps have issues - write once, run anywhere (as long as there are no spaces in the path). Other commenters chimed in issues with *nix application ports having similar path issues. (My peeve #19).

Foxyshadis points out a few more: "are you sure you want to create the folder" AND "are you sure you want to install in an existing folder" - I don't get this one either (My peeve #10). "Once you install with custom options and later upgrade, and try to use typical, not only will it always forget which options you chose, it'll install into a completely new default folder." Why is that? This is a nice addition to my peeve #11.

Gryphonvere points out some applications "ask you rather forcefully to install some older version of DirectX" - some of this is the fault of the dependent application not providing a mechanism to determine the current version of itself, and some on the setup developer for not doing the smart thing and saying "This application was not tested with Acrobat Reader 12, if you have issues, try using Acrobat Reader 4.0." This is a corollary to my peeve's #17 and #13.

David Walker complains about the "Company Name\Product Name" default directory convention (My peeve #14). He also hits the "Common Files" directory issue. This latter point is somewhat valid, but for keeping the redistribution, duplicate files, and simplified patching for a company's product line shared components down to a minimum, I can live with that. We all have to understand that x-copy deployment of applications or product lines is not always possible. While on the subject (and expanding on my earlier peeve #20), my belief is that "Application Data" folders are underused. If I want to back up my system or use OS features such as roaming profiles, I want to have a one-stop method of doing so. Backing up the per-user application data folder, the per-system application data folder, and the "My documents" folders should be the only backup philosophy I ever have to use.

Doesn't anyone test the key features of an installation for both correctness and usability anymore?

Application Preloading at System Startup

Sometimes I wonder if we (as software developers) should have some form of a code of ethics, where violations can get you barred from writing software in the future. Mainly, this would apply to the spyware/malware developers (or perhaps the ethical violation is more directable to those who bundle the spyware).

If this ethical code existed, I would add a violation for any application that did not inform and offer a choice to the user at install time that some startup or background task will always be running. Some typical offenders are QuickTime, Real, Adobe, Microsoft Office, Microsoft Messenger, printer driver "status monitors", and more. These startup applications increase boot time and memory/pagefile requirements. Most of the time, I don't use these applications each time I boot the PC.

If the goal is to make your application start up faster (or appear to start up faster) there are ways of accomplishing this. Rebasing your DLL's, background/on demand loading of plugins, delay loading dependent DLL's, etc. are a few common tricks that can be used. The utility "Adobe Reader SpeedUp" accomplishes a magnificent improvement in load times, and this doesn't involve rewriting any of Adobe's code. If you wrote one of these startup applications anyway, at least offer the user the option at install time if they wish to use it.

Checking or Changing System Settings

Raymond Chen reminded me of a story I wanted to share. In non-MSI setups (and I am only using setups as an example of a larger problem in application programming), installing a service was a bit strange and usually flaky. Some installation engines would simply write the service registry keys directly into the "CurrentControlSet" section of the registry and require a reboot for the service to work. The reason was the Service Control Manager only looks at that portion of the registry when the machine is starting. Some engines decided to write values to the HKLM\System\ControlSet001 key - an even worse idea. Recall that "CurrentControlSet" does not really exist, it is mapped to the "real" ControlSet001 or ControlSet002 based on how the machine was booted. This is sort of like the HKEY_CLASSES_ROOT virtual key I blogged about before. If you need to query information about services, install them, etc. use the API's intended for this purpose - not the registry keys. Not only is this good advice for services, but in general. If you are using a registry key to check or change a setting, check for an API to set/check for the setting first. This generally isolates you from potential incompatibilities across future OS versions.

Raymond's post relates to User Interface settings, and describes an application that crashed under Windows 2000 because of the reliance on registry keys and not API's. Rephrasing my earlier point - if there is an API that reads or writes registry key data, it should be considered "Private".

One commenter to Raymond's post responded "sometimes its extremely hard to find the right api, with so many apis hanging around" - and he is 100% correct. This is where having experienced people around your development shop helps. If you don't have the former, this is a case where reading portions of the MSDN library (and blogs like Raymond's and Larry's) helps. If you are short a photographic memory, you won't retain everything - but you may remember reading something in the past and a few keywords that you can Google for later. Microsoft can help programmers avoid this trap somewhat by adding the keys that are read from or changed by API's in the MSDN documentation with a disclaimer - "The following key stores data related to this API but should not be relied upon as an implementation detail in future OS releases."

Sometimes it is hard to follow these "rules." Take for instance the Distributed Transaction Coordinator, or DTC for short. If you wanted to get or change any particular setting related to DTC you can't do it the same way across OS versions. In Windows 2003, there is the IDtcNetworkAccessConfig interface you can use. To change the same settings on Windows XP, the registry appears the only option. Furthermore, the upcoming Windows 2003 SP1 DTC changes (which mirror the XP SP2 changes) has not updated the API to reflect the change (at least in the SP1 RC2 version of the SDK documentation). Specifically, there is no updated method of allowing or disallowing specific transaction permissions such as inbound or outbound, so the API is useless - back to registry keys! Checking the DTC authentication level never had an API - just some registry keys documented in a KB article. I hope they fix this oversight for the final SP1 release - but in the meantime, what is a developer expected to do?

Active Setup Registry Keys and their Purpose

I’ve been very busy lately, and I’m sorry about the blog being so quiet. I haven’t gotten around to completing the Custom Action DLL tutorial, but I’m pretty close. I need to find a server that would let me store some zip files. Any ideas?

Back in September, Aaron Stebner blogged about detecting what .NET 1.0 Service Pack is installed. One of the keys he mentions checking has the word "Active Setup" in it. Just what is that key for?

The Active Setup key in Local Machine is read when a user logs in to the system. Keys in the Local Machine registry hive are compared against keys in the Current User hive. If a key that exists in Local Machine does not exist in the Current User hive, the program the key points to is run. Effectively, this is one way to customize (or completely remove) an installation on a per-user basis, assuring that a program is executed exactly once per user.

There are several installations that use this key. Microsoft Net Meeting and Internet Explorer are a few examples. Lets say you update Internet Explorer to a new version – like IE 5 to IE 6 – and you have two active user profiles. When the computer is restarted and the user logs in, you see a dialog that shows IE is being configured. When the other user logs in, they see the same dialog. This is Active Setup at work! Net Meeting uses this mechanism to cleanup user profiles after Net Meeting is uninstalled. Sadly, virus and spyware packages also use this mechanism (changing the Local Machine key after each reboot), forcing the vigilant advanced user to check yet another key for items that run at startup. This is yet another reason to run as a restricted user!

The limitations of this mechanism are simple: The program that is run from Active Setup runs in the current user space. Therefore, to be completely safe, any Active Setup program should require read/write access of the most restricted user type – only modifying files and registry keys owned by an individual user. Additionally, this is run BEFORE the shell and other run keys.

There is no "official" documentation on these keys or this behavior, so the normal caveats apply to using this mechanism – it may not be present in newer versions of the OS or Service Packs.

Here is some detailed descriptions of the keys and their contents. A registry key called "HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components" has a bunch of subkeys under it that are GUIDs. Values contained under the GUID key that we are talking about are "Version" (string), and "StubPath" (Expand String).

Every time a user logs in, the contents of this key is compared against the SAME key in HKCU. If the HKCU key does not exist and/or the "version" value is less than that in HLKM, the appropriate "StubPath" command is run and the key copied to HKCU so it is not run again.

To see how this works for yourself, you can create a key in HKLM…Installed Components and call it "test". Then, add to it a String Value "StubPath" and set it to "notepad". Reboot. See that when you log in, notepad starts. Log in as a different user. See that notepad starts. Log in again with the first ID – notice notepad does not start – as now the "test" key has been copied into the HKCU branch after it ran the first time. Note that version is not necessary.

Since I am not sure what the other values of these keys mean and/or do, I’m going to end the discussion here. If anyone has more information on these keys, please add it to the comments for this post!

Insane security

Raymond Chen posted about some questions better left unanswered because the answer is unreliable. This quote cracked me up:

"You cannot reliably reason about the security of a system from within the system itself. It's like trying to prove to yourself that you aren't insane."

Although the quote is funny, he makes several points about security that developers should heed. Commenters have taken the post a bit out of context - but the side point he makes is valid. Since most of my audience is setup/installation developers, I'll direct my comments towards that community.

The next time you are given a design document for installing a service, please make sure to ask about the security level required for the service. Make sure to raise objections to installing a service with access to the desktop without good reason (read comments to this post for more information related to the shatter attack) . Ask if LocalSystem access for the service can be reduced to a more restrictive account (HINT: There is a sample for creating user accounts within an MSI in the Platform SDK). Most of all, insist that scoping of installation/deployment design is best when it is done with the scoping of the feature or application - not after. This way, the application developer can develop in an environment closest to that which will ultimately be deployed to customers in the installation with no surprises looming at the end of a development cycle.

In case you haven't noticed, Windows XP SP2 and Windows Server 2003 have tightened up security in these areas. Instead of blindly loosening up the default restrictions this brings, take the application to the next level by understanding why the restriction is now the default in the first place, and learn how to program taking into account this more secure environment. As a setup developer, your customer is the end user. Make sure you are on the side of the customer when it comes to security.

Raymond is on a roll... (shared files, versioning and setup)

Fellow blogger Raymond Chen has posted yet another blog of note to setup developers. I'm going to use a different example than he did, plus use it as a springboard for yet another rant. If you remember back to Windows 95, especially when configuring network settings, it would prompt for the CD (bad) and then prompt to if you wanted to overwrite a newer version of a file with an older version (even worse). As an added bonus, it asked this question more than once (the absolute worst).

Why did it do this? Read Raymond's answer. For this specific example, most people installed updates to the original TCP/IP stack. Windows 95 wanted to restore the initial version of all files in all cases. This could leave the user hosed if they didn't answer the questions consistently. Microsoft learned from this example. We should, too.

Lets look at how the MSI engine handles patching. OK, maybe that would be too long of a rant:) The essential thing to take away is that a MSI patch (msp file) will not overwrite a newer version of a file with an older version. In the usual case, this is not a problem, since all identically named DLL's are always backwards compatible (in Utopian software development), right? This is a decision the Windows Installer engine designers had to make up-front. And as long as we are in Utopia, it works.

If you recall, many older InstallShield installations also prompted the user on uninstall if they wanted to remove a DLL that is marked as shared, since no other programs claim they are using it. The more modern versions of InstallShield don't. Why? They expect us (setup developers) to get it right. Make sure you are using the appropriate method of installing files that may be shared by other installations. In the MSI world, Merge Modules accomplish this. But especially for non-MSI installations, make sure you are setting the "shared" flag, so the uninstallation of you application does not impact other installed applications by removing files critical to their operation.

As a side note, Raymond links to an older article related to asking useless questions that is quite humorous, yet further proves this important point: If you ask the user a technical question, odds are they they're just going to stare at it blankly for a while, then try to cancel out of it.

Don't make these mistakes.

Control Panel Items

Since I'm on a roll with configuring stuff, there is a nice article on MSDN about configuring Control Panel Items. If you are a setup developer, this link will take you to the section that you should read.

Some things to note:

• "...the DLL file must have a .cpl file name extension" - These control panel items ARE DLL's.
• on "Windows 2000 and later systems, new Control Panel items should be installed in the associated application's folder." - Yep, no installation to SYSTEM like the old days!
• The placement of a registry key dictates what shows up in the Control Panel. Remember to respect per-user installations! (Yes! it is a bug, a security one, and a bad one)
• If you must support the "old" pre-Windows 2000 method, only do so on the pre-2000 OS.
• Yes. you can extend System Control Panel Items.
• You can also specify a category for the item in Windows XP or better, for users using the default control panel view.
• For MSI folks, there is nothing in any of the Standard Actions that will help you.

Larry talks HKCR

If you recall, I recently blogged about the fact that HKEY_CLASSES_ROOT key is not a real registry hive. Larry Osterman from Microsoft recently posted a "What is wrong with this code?" series and covers the HKCR issue extremely well in the answer section. If you develop services, this one is worth a read.

Raymond talks about registering shell extensions

As you can tell, I'm catching up on reading some of my favorite blogs... Raymond Chen from Microsoft wrote a really nice article about registering shell extensions. Setup developers should take note of how Windows deals with what you put in those keys.

What is a shell extension (also known as Verbs)? This is what Windows uses to determine what to do (or what right-click options to provide) when clicking on a file with a specific extension. Read this link to find our more. I'll wait...

In the setup world, I believe when you run a program from a setup most vendors use the ShellExecute() API. If this is the case, when you launch a readme at the end of an installation by simply specifying "readme.txt" the default "open" verb (on a fresh Windows installation for a text file this will be Notepad) will be run. If the method of running an application in your authoring tool uses CreateProcess() , it will likely result in an error if you simply give it "readme.txt". For MSI based installations, many people may use the tutorial method described in the Platform SDK. I leave it as an exercise to the reader to see what that one uses! Any guesses based on the specifications for the tutorial mentioned in this document?

If you read the MSDN documentation for these functions - which are linked to above - you will understand this quote from the article: "Consequently, the shell mistakenly believes that the program name is "C:\Program", which it cannot find." This can be a security issue if a malicious program installs itself as "C:\program.exe". Please make sure you are registering extensions correctly - even if the installation design document you are given dictates otherwise.

.NET Deployment

I came across this blog entry that talks about this blog entry. So this blog entry is talking about a blog entry that is talking about a blog entry - whew!

The crux of the story is how an application based on the .NET Framework should be deployed. The post goes into adding the MSM for the framework, etc. The killer quote from the first blog entry by Stéphane Rodriguez is "What should I, as a developer, think about a development framework which by design involves hacks only for my app to be deployed?"

Amen, sister!

I blogged previously about how difficult it is to determine the .NET Framework and again when Aaron Stebner produced some sample code to check this. Stéphane hit exactly on what was in the back of my mind: Why is this so hard?

In theory, the .NET runtimes are mostly backwards compatible. The version detection scheme is not future compatible - I have no idea how to detect the .NET 2.0 RTM runtime today as a setup developer. Even if I can detect this, there are no guarantees that "Compatibility and Breaking Changes" haven't affected me. At least if I could detect the version, I could intelligently deal with it. I may alert the user that the framework version installed has not been tested with my application and let the installation continue. Then again, I may choose to have the old framework downloaded and installed. Or really anything.

The problem that exists is (as a commenter to the quoted blog points out) very similar to Java's problems. Microsoft should have learned from this and done a better job. Looks like they haven't - and mostly from a documentation perspective from what I can tell.

Lastly, there is a lack of tools and guidance to setup developers and how they should handle deployment. In an MSI, why can't I simply tell the engine/tools that my application targets one or more versions, and let the engine handle the detection job along with a warning if installed framework version is greater than what is targeted?

Why must stuff that should be so easy be so difficult?

GDI Plus Vunerability Information for Setup Developers

Stefan Krueger from InstallSite.org has posted a very well-written article about the GDI+ vunerability and how setup developers should respond to it. Please read it ASAP, it is a great consolidation of all the available information, and should be acted upon quickly.

If you haven't subscribed to Stefan's RSS feed, here is the link. If you don't know what RSS is, try this link.

Top 20 Installation Pet Peeves and How to Avoid Making Them

This Top-20 list of pet-peeves/annoyances is presented in no specific order and pertains to all installations, however, some items and examples directly relate to MSI based installations.

Am I missing your top pet-peeves? Are you guilty of any of the below and care to admit it? Please leave feedback in the comments!

20. Security and profile unaware installations. Can the application be installed or used by a non-administrator? Use the principle of least-privilege to ascertain what the real limitations are and make installations compliant with it. Respect user profile settings. Is the installation storing sensitive information unencrypted (such as passwords) in your log files, registry/INI files, or MSI files?

19. Installing to a location other than "Program Files" by default. This also applies to installations that assume "Program Files" is located on the C:\ drive. I recommend testing installations and applications on systems that have Windows installed to a drive other than the default C: drive with a Windows directory name other than Windows or WINNT to catch these assumptions.

18. Non-application file storing directories sharing the application directory. No application should store configuration information or user data (by default) to a directory other than the current user's documents and settings directory. Similarly, if the installation is creating a shared folder, virtual directory, or a directory that will have security implications/working data in it (such as file-sharing or database), the user should be able to select this directory separately (and it should be defaulted to a non-Program Files directory). The rationale for this is simple: Applications are restorable by reinstalling the software. User data isn't. Make it easy for the user to back up these items by default.

17. Not providing a merge module if this software will be redistributed by other vendors. I absolutely despise running an executable from my installation to install a 3rd party component. Not only do I have no idea if the sub-installation succeeded or failed, but automatic reference counting is impossible. If I repackage it, other vendors could do the same and we have out of whack component codes and keyfiles causing endless automatic repair cycles, maintenance, and removal nightmares. This also adds an easy way to assure a common method of distributing patches if the third-party software is found to have a security vulnerability.

16. Full screen installations are evil. Why does the entire screen need to be used for an install whose UI is 640x480 or less? Can't I do anything else while the install is in progress? Conversely, if the install requires a screen resolution greater than 800 x 600, my parents can't install it (or worse yet your help desk will be overwhelmed by my parents and their friends).

15. Product Serial Numbers are usually a pain. If the installation is using the Microsoft style dash separated product keys, don't also suffer from same problem that prevents a simple one step cut and paste from a registration email into the install UI. Also, try to prevent the use of commonly confused digits in the registration code, as it is font and eye-strength dependent when trying to figure out if the code is a capital O or a 0, a lower case l or a 1. If millions of codes are not needed, consider axing the U and V as well.

14. Multiple level start menu shortcuts. If a folder in the start menu only has one option in it, please do not put it into a folder. Also, make sure the folder name makes sense to the majority of end users. If I'm looking for a program "Ink Blot" am I going to think to look in the folder "Joe Schmo Software" first?

13. Unintelligent replacement of system or other redistributable files. Are you using a merge module for redistributables if one exists? Does the install check file versions/dates before replacing them? Is reference counting (shared dlls for legacy installations) being set properly? Are all required redistributables included?

12. Unnecessary Reboots. The converse is also true, where reboots are required but not asked for. Also in this category is detection of running software that makes a reboot necessary prior to running the guts of the install. Please let the user know if they can do something that will prevent a reboot at the end. If it is an older version of the installed application that is running that will require the reboot, do what is required to prevent the reboot (as long as you tell the user first). See Pet-Peeve #3.

11. Upgrades that require an uninstallation of the old version first. If this is truly required, is it being handled automatically?

10. Asking useless questions. If the installation behaves identically regardless of the install type (minimum/typical/custom) why is that dialog there? If there is a difference, is the text on that dialog accurate and/or does it make sense in the context of your application? Also in this category are error messages such as "Unable to do something important. Would you like to continue?" when "something important" is essential to the functionality of the application. This can also extend to the age-old useless "shared file no longer in use, should I delete" question.

9. Status bars that are not accurate (or present). I realize that writing status bar code is incredibly difficult. If we are talking InstallScript, VBScript, or JScript installations/actions threading is impossible, so the author can't give a moving status bar when performing a synchronous operation (like creating a database). If an alternate method is not possible, another strategy is to put a message up saying "Busy doing [something long] which may take up to 10 minutes, started at xx:xx". If you are using a C++ custom action that can be threaded to perform the synchronous action, determine the maximum time on the slowest PC meeting your system requirements and provide a time-based incrementing progress bar.

8. Allowing free-text user inputs when choices can be given. As an example, let's say the installation upgrades a database. Presenting a free-text input box for the old database name is a recipe for disaster. Instead, the install can perform a SQL query of the database server to get a list of databases. If the upgrade is applicable only to a specific type or version of database, it can then query each existing database to determine which ones the upgrade can be performed on and populate a selection list. In silent mode, any UI level checks will not be run, so validation should be performed on any command line parameters as well. See pet-peeve #1 and #6.

7. Requests for source media. If I am on my laptop (and more people are each day), and especially if I am out of the office (or a remote employee) it is practically impossible to get the source media at a moment's notice. This is especially true for installations of patches where a security vulnerability is in the wild and users need to patch immediately. Make sure nothing is being done that would prompt for source media if possible, and try to assure that the MSI and support files is going to be available by caching locally by default. Office 2003 made significant headway in this department.

6. Non-working silent installations. Make sure silent installations work properly. This includes checking properties set on the command line as you would user input in the UI, and failing with some meaningful error posted to a log if required parameters are bad or not present. Are all user interface elements able to be passed in via the command line? Failure to provide a silent installation means IT departments will grumble heavily or look for an alternative if they need to deploy your application.

5. Repair/Maintenance/Uninstallation/Rollback doesn't work. If the install is not going to provide anything useful for maintenance mode, get rid of the option. Make sure all custom setup properties are cached somewhere and pulled back in for the repair operation to succeed. Does auto-repair work correctly (delete the install directory, run from the shortcut, and see if it works)? Does an uninstallation remove everything, including cached copies of the installation? Does an error near or at the end of the installation properly rollback the changes (Throw in a deferred custom action that simply fails prior to InstallFinalize)? If altering a DB, is it being backed up before running the portions of the installation that alter it so the previous version can be restored in a failure case?

4. Assumptions that are not checked. If an installation adds a database to SQL Server 2000, is the system checked for the presence of a running SQL server before it allows the user to start the installation? Think about a system that has SQL Server installed, but the service set to disabled or is currently stopped. Similar comparisons can be made to installations that rely on web servers, messenger service, BITS, etc. If the service is not started or set to disabled, is the installation violating pet-peeve #3?

3. Modifying a system globally without asking or telling the user. This is a huge category with security implications. If the installation adds a network share or internet accessible directory, is the user being told this up-front? Take as an example the .NET framework installation. Is the user asked/told if I want ASP.NET installed to the IIS server? The answer is no. A mention of this or other types of global system changes in the documentation alone (especially since most documentation is installed with the product) is unacceptable. Note that most modern media player installations ask the user if he or she wants to associate file types with it. Installations were not always this nice to your system.

2. Not checking return codes. When running a command that is supposed to do something essential for a successful installation, does the installation check the return code and handle the error? A great example of this problem is found in many database installations, where the result of a SQL script is not checked. The application does not work after a reported "successful" installation, and it takes a tremendous amount of time and effort to determine why.

1. Not validating user input. The install allows the user to free-text something in where choices can not be predetermined (see pet-peeve #8); let's say a timeout value or something that is essential for the installation to succeed or the application to function. The user types in "10h" by accident. The install doesn't verify the input is a number or a number within a valid range. Installation fails (bad) or says it is successful, yet the application does not work (infinitely worse).

Aside from this list, a very useful document for installation (and application development)authors to follow is the "Designed for Microsoft Windows XP" Application Specifications. Many of the items listed there are also listed above.

Detecting .NET Framework Version (Update)

Aaron Stebner from Microsoft comes through with some source code (in C++) to determine the installed version of the .NET framework, including service packs. Great job, Aaron!

Detecting .NET Framework 1.1 SP1

Aaron Stebner from Microsoft blogged about how to determine if the 1.1 version of the .NET framework is installed. Unfortunately, that is just to detect if 1.1 is installed.

One method of checking that Aaron quoted is described by a Microsoft Knowledge Base article. This approach is impossible to perform programmaticly. He also gives a registry key to check for the 1.1 .NET framework, but is a bit unclear if this key is for 1.1 or 1.1 with the service pack.

Other methods of checking I uncovered are difficult to do in code in a manner that makes sense to work in the generic case. The confusion that led to Aaron's post in the first place are questions (and the answers/lack of answers) posed in newsgroups, like this one.

The .NET framework version checking problem reminds me of old InstallShield code. I can't recall if this was built into InstallShield or if it was a bad code sample, but lots of installs failed on Windows 2000 simply because the install code checked to see if you were NT 4.0 SP3 or higher. The detection code was not based on proper logic, but rather if OS version >= 4 and SP version < 3, do not install. What a headache (at least until Windows 2000 SP3 came out)!

Fortunately, Aaron is going to research getting a code sample to detect the old versions and hopefully Microsoft can provide a future-proof method to determine this in upcoming revisions.

The Case of the Disappearing DLL

Although this is not quite the Sherlock Holmes and Dr. Watson type of mystery, this bug was an interesting one to troubleshoot. After reading the bug report, I suspected what the problem was, but could not reproduce it on purpose in the lab without some hassle.

The bug report stated that some of the DLL's required by our product were deleting themselves. After installation, everything was fine. Sometime weeks later, a DLL or two mysteriously vanished.

The culprit? On an uninstall of the old version of the software, a DLL was in use, and its removal slated for post reboot. After the uninstall ran, the user promptly ignores the reboot request and installs the new version of the product. The dll is no longer in use at this point, so the installation succeeds and no reboot is prompted for. The next time the user rebooted - blammo - bye, bye DLLs!

I'm not going to give away my solution for this one (other than it involved fixing the uninstall process going forward), but one possible way to prevent this is to disallow installations if there are pending file removals or replacements. You can check the registry at HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations to see if there is some activity scheduled for after reboot. There is a tool that dumps this information and another one that captures that plus some other post-reboot activities. One of these tools should be part of every setup developer's toolbox.