Debugging DLL Registration Issues

Did you know that the HKEY_CLASSES_ROOT registry hive (in Windows 2000 and above) is a figment of your imagination?

I was helping a coworker troubleshoot an application that runs as a service that accesses another homebrewed COM component. Of course, it worked on his machine, but not when deployed on a test machine. Running the normal Filemon/Regmon tag team, we traced it down to an issue instantiating the COM component. Regmon reported a registry key in HKEY_CLASSES_ROOT could not be found. Double clicking the line in Regmon (I really love that feature) opens up Regedit to the key that couldn't be found. I immediately guessed the root cause of the problem, and asked the developer the question above: Did you know that the HKEY_CLASSES_ROOT registry hive (in Windows 2000 and above) is a figment of your imagination?

He responded, "Huh? Its right there," as he pointed to the topmost hive in Regedit. A quick pointer to MSDN followed by browsing to HKEY_CURRENT_USER\Software\Classes and HKEY_LOCAL_MACHINE\Software\Classes showed the obvious - the key that Regmon showed as "not found" was really in the HKEY_CURRENT_USER section. As you may know, an application that runs as a service is run under the LocalSystem account (assuming no impersonation is done in the service, a user is not specified in the Service Control Manager, etc.). This account does not share the HKEY_CURRENT_USER hive with the user that installed the software, nor the currently logged in user, and was therefore unable to read the registry key necessary to instantiate the COM component since it did not exist for LocalSystem.

So what caused the problem? In an MSI created by most authoring tools, an option is given to the user to install the application for the "current user" or "all users". This ultimately sets the ALLUSERS property. In addition, the Windows Installer engine will add the registration information to either the HKCU or HKLM sections as defined by the ALLUSERS setting. The only documentation related to this behavior is buried in the SelfReg table, but not in the documentation for the property, tables, or actions that are truly affected by this. Bear in mind that for items in the SelfReg table, the DllRegisterServer() function is called in the context of the current user if ALLUSERS is set to a "this user only" setting, and "elevated" if it is a per-machine installation.

It is tempting to skip the bland and usually useless "User Information" dialog when creating your installations so that your installation doesn't get on people's nerves, but remember that while setting this property is important, you surely do not want to violate rule #20 while trying to appease rule #10.

Oh, and if you are interested in how I actually fixed this issue: since the per-user installation makes no sense for this application, and the "User Information" dialog was never shown, a custom action of type 51 came in handy to set ALLUSERS to 1 at the start of both the UI and InstallExecute sequences. Some ICE type validators may flag the installation if ALLUSERS is set in the property table. If you forcibly set ALLUSERS to 1, make sure that the LaunchCondition table checks that the user is Privileged and that Version9X is not set.

Wix video presentation

Rob Mensching published a video of a presentation he gave back in June regarding the Open Source Wix MSI toolset.

The video is a 64 MB download. My recommendation - do a right-click and SaveAs on the link.

Running as Non-administrator

Along with the recent security frenzy, many people have suggested running as a non-administrator on Windows 2000/XP. I attended an MSDN event for developers dealing with ASP.net several months ago, and was surprised that the presenter was running as non-admin. He also gave some great tips on how to make it easier.

First off, you should read Aaron Margosis' blog in its entirety. If you need convinced that you should run as a non-admin, start with this post. Then read all the others. They start back in June. If you already read them, skip the rest of this paragraph, as I will summarize the info you can get there. I like the "easiest way" for non-techie people. Its rather simple, but effective. Pay attention to the "update" mentioned, where he suggests making the admin desktop easily distinguished from the non-admin desktops. For the more tech-savvy, I prefer the shortcut with RunAs setup. I change the color of windows such as explorer, when running them as administrator, and suggest you do the same. Alternatively, PrivBar can be used, but I have not installed or used it myself.

Ok, so it should be about 15-20 minutes later since you read all the above stuff, right? Normal people can go off and read their next blog now after they made the necessary changes to their system - but I have a special message for you developers...

I blogged about pet peeves in installation programs and applications. Refresh your memory regarding items #20, 19, and 18. (Do you see a trend? I already covered #9 in a previous post). Hopefully you are starting to see another connection forming between non-admin users and where installations and applications put and use their files. To reinforce this connection read Larry Osterman's post from today talking about running as non-admin. This stuff is applicable to us.

Make sure your testing staff is running as non-admin to catch these issues. Not mentioned in the article is that you should make sure you are using XP Professional with NTFS disks so you can catch the strange permissions problems that can add.

Other test paths include installing the application as administrator for "all users" and logging in as a different, unprivileged user and make sure everything operates correctly. MSI Developers - don't ignore the ICE warning about mixing per-user and per-machine entries in a single component. Lastly, make sure the settings in your application are user profile aware. I recall trying to figure out how to make a media player work on my Uncle's PC with different profiles. For good reason, my uncle didn't want to share a playlist with his son - but that wasn't possible. Bye-bye old software vendor, hello new software vendor...

If you are a developer that doesn't work for Microsoft, you can go onto the next blog now, unless you are a real glutton for punishment:)

At my company, the IT department is pretty lenient to the developers. They give us admin privs to our own machines. The problem is I don't want admin privileges all the time, just when I'm doing stuff that requires it. One solution is to give me a local account admin account to use when I need to admin my box, and only normal rights at the domain level. The problem with this approach is I can't access network shares using my domain account when I am running as local admin for tasks like installing software. Having two domain accounts is a nightmare. I don't know what the solution is - perhaps a flag to make the account have normal user privs, and when administrative privileges are required have the security subsystem prompt me for explicit permission to elevate that process. I guess I'd call it the "Admin Masquerade" group for lack of a better term.

GDI Plus Vunerability Information for Setup Developers

Stefan Krueger from InstallSite.org has posted a very well-written article about the GDI+ vunerability and how setup developers should respond to it. Please read it ASAP, it is a great consolidation of all the available information, and should be acted upon quickly.

If you haven't subscribed to Stefan's RSS feed, here is the link. If you don't know what RSS is, try this link.

Making Progress Bars behave nicely

After my previous blog entry about installation pet-peeves, including how writing progress bar code is hard, I figured I would try to help a bit by providing a simple sample for MSI authors. At the same time, a question related to progress bars appears on the newsgroups. Coincidence?

If any of you have some stylesheet stuff I can add to make posting/reading code snippets on a blog less painful for both you and me, please send it in.

The usage in a Custom Action is pretty simple: you parse CustomActionData if you are running deferred and determine how many things you have to do. Then call setupProgress("MyCA","Transmorgifying files",10); if there are 10 things to do. When performing each of the 10 things, shoot off a call to tickProgress("this filename"); and the bar should behave nicely. It's pretty easy to convert to the Custom Action language of your choice. Most of the information needed to create this sample is in the Session.Message() documentation.

var msiMessageTypeProgress = 0x0A000000;
var msiMessageTypeActionStart = 0x08000000;
var msiMessageTypeActionData = 0x09000000;

function setupProgress(ActionName, ActionDescription, numTicksExpected)
{
//set the UI Status ACTIONSTART
var statusRecordObj = Installer.CreateRecord(3);
statusRecordObj.StringData(1) = ActionName;
statusRecordObj.StringData(2) = ActionDescription;
//below line defines a template for display on each message tick
statusRecordObj.StringData(3) = "[1]";
Session.Message(msiMessageTypeActionStart, statusRecordObj);

//resets the progress bar.
var progressRecordObj = Installer.CreateRecord(4);
progressRecordObj.IntegerData(1) = 0; //0 = reset bar
progressRecordObj.IntegerData(2) = numTicksExpected+1;
progressRecordObj.IntegerData(3) = 0; //0 = left-to-right, 1= R-to-L
progressRecordObj.IntegerData(4) = 0; //0 = calc time remaining
Session.Message(msiMessageTypeProgress, progressRecordObj);

//tell the progress bar to increment for each ActionData message
progressRecordObj.IntegerData(1) = 1; //Action Info
progressRecordObj.IntegerData(2) = 1; //numTicks each ActionData
progressRecordObj.IntegerData(3) = 1; //increment bar for each ActionData
Session.Message(msiMessageTypeProgress, progressRecordObj);
}

function tickProgress(message)
{
//The ActionData record follows the template defined in the msiMessageTypeActionStart
//item #3. Refer to the MSI documentation related to templates. If you add more fields
//to the template, make sure to add the additional fields here as well.
var progressObject = Installer.CreateRecord(1);
progressObject.StringData(1) = message;
Session.Message(msiMessageTypeActionData, progressObject);
}

Top 20 Installation Pet Peeves and How to Avoid Making Them

This Top-20 list of pet-peeves/annoyances is presented in no specific order and pertains to all installations, however, some items and examples directly relate to MSI based installations.

Am I missing your top pet-peeves? Are you guilty of any of the below and care to admit it? Please leave feedback in the comments!

20. Security and profile unaware installations. Can the application be installed or used by a non-administrator? Use the principle of least-privilege to ascertain what the real limitations are and make installations compliant with it. Respect user profile settings. Is the installation storing sensitive information unencrypted (such as passwords) in your log files, registry/INI files, or MSI files?

19. Installing to a location other than "Program Files" by default. This also applies to installations that assume "Program Files" is located on the C:\ drive. I recommend testing installations and applications on systems that have Windows installed to a drive other than the default C: drive with a Windows directory name other than Windows or WINNT to catch these assumptions.

18. Non-application file storing directories sharing the application directory. No application should store configuration information or user data (by default) to a directory other than the current user's documents and settings directory. Similarly, if the installation is creating a shared folder, virtual directory, or a directory that will have security implications/working data in it (such as file-sharing or database), the user should be able to select this directory separately (and it should be defaulted to a non-Program Files directory). The rationale for this is simple: Applications are restorable by reinstalling the software. User data isn't. Make it easy for the user to back up these items by default.

17. Not providing a merge module if this software will be redistributed by other vendors. I absolutely despise running an executable from my installation to install a 3rd party component. Not only do I have no idea if the sub-installation succeeded or failed, but automatic reference counting is impossible. If I repackage it, other vendors could do the same and we have out of whack component codes and keyfiles causing endless automatic repair cycles, maintenance, and removal nightmares. This also adds an easy way to assure a common method of distributing patches if the third-party software is found to have a security vulnerability.

16. Full screen installations are evil. Why does the entire screen need to be used for an install whose UI is 640x480 or less? Can't I do anything else while the install is in progress? Conversely, if the install requires a screen resolution greater than 800 x 600, my parents can't install it (or worse yet your help desk will be overwhelmed by my parents and their friends).

15. Product Serial Numbers are usually a pain. If the installation is using the Microsoft style dash separated product keys, don't also suffer from same problem that prevents a simple one step cut and paste from a registration email into the install UI. Also, try to prevent the use of commonly confused digits in the registration code, as it is font and eye-strength dependent when trying to figure out if the code is a capital O or a 0, a lower case l or a 1. If millions of codes are not needed, consider axing the U and V as well.

14. Multiple level start menu shortcuts. If a folder in the start menu only has one option in it, please do not put it into a folder. Also, make sure the folder name makes sense to the majority of end users. If I'm looking for a program "Ink Blot" am I going to think to look in the folder "Joe Schmo Software" first?

13. Unintelligent replacement of system or other redistributable files. Are you using a merge module for redistributables if one exists? Does the install check file versions/dates before replacing them? Is reference counting (shared dlls for legacy installations) being set properly? Are all required redistributables included?

12. Unnecessary Reboots. The converse is also true, where reboots are required but not asked for. Also in this category is detection of running software that makes a reboot necessary prior to running the guts of the install. Please let the user know if they can do something that will prevent a reboot at the end. If it is an older version of the installed application that is running that will require the reboot, do what is required to prevent the reboot (as long as you tell the user first). See Pet-Peeve #3.

11. Upgrades that require an uninstallation of the old version first. If this is truly required, is it being handled automatically?

10. Asking useless questions. If the installation behaves identically regardless of the install type (minimum/typical/custom) why is that dialog there? If there is a difference, is the text on that dialog accurate and/or does it make sense in the context of your application? Also in this category are error messages such as "Unable to do something important. Would you like to continue?" when "something important" is essential to the functionality of the application. This can also extend to the age-old useless "shared file no longer in use, should I delete" question.

9. Status bars that are not accurate (or present). I realize that writing status bar code is incredibly difficult. If we are talking InstallScript, VBScript, or JScript installations/actions threading is impossible, so the author can't give a moving status bar when performing a synchronous operation (like creating a database). If an alternate method is not possible, another strategy is to put a message up saying "Busy doing [something long] which may take up to 10 minutes, started at xx:xx". If you are using a C++ custom action that can be threaded to perform the synchronous action, determine the maximum time on the slowest PC meeting your system requirements and provide a time-based incrementing progress bar.

8. Allowing free-text user inputs when choices can be given. As an example, let's say the installation upgrades a database. Presenting a free-text input box for the old database name is a recipe for disaster. Instead, the install can perform a SQL query of the database server to get a list of databases. If the upgrade is applicable only to a specific type or version of database, it can then query each existing database to determine which ones the upgrade can be performed on and populate a selection list. In silent mode, any UI level checks will not be run, so validation should be performed on any command line parameters as well. See pet-peeve #1 and #6.

7. Requests for source media. If I am on my laptop (and more people are each day), and especially if I am out of the office (or a remote employee) it is practically impossible to get the source media at a moment's notice. This is especially true for installations of patches where a security vulnerability is in the wild and users need to patch immediately. Make sure nothing is being done that would prompt for source media if possible, and try to assure that the MSI and support files is going to be available by caching locally by default. Office 2003 made significant headway in this department.

6. Non-working silent installations. Make sure silent installations work properly. This includes checking properties set on the command line as you would user input in the UI, and failing with some meaningful error posted to a log if required parameters are bad or not present. Are all user interface elements able to be passed in via the command line? Failure to provide a silent installation means IT departments will grumble heavily or look for an alternative if they need to deploy your application.

5. Repair/Maintenance/Uninstallation/Rollback doesn't work. If the install is not going to provide anything useful for maintenance mode, get rid of the option. Make sure all custom setup properties are cached somewhere and pulled back in for the repair operation to succeed. Does auto-repair work correctly (delete the install directory, run from the shortcut, and see if it works)? Does an uninstallation remove everything, including cached copies of the installation? Does an error near or at the end of the installation properly rollback the changes (Throw in a deferred custom action that simply fails prior to InstallFinalize)? If altering a DB, is it being backed up before running the portions of the installation that alter it so the previous version can be restored in a failure case?

4. Assumptions that are not checked. If an installation adds a database to SQL Server 2000, is the system checked for the presence of a running SQL server before it allows the user to start the installation? Think about a system that has SQL Server installed, but the service set to disabled or is currently stopped. Similar comparisons can be made to installations that rely on web servers, messenger service, BITS, etc. If the service is not started or set to disabled, is the installation violating pet-peeve #3?

3. Modifying a system globally without asking or telling the user. This is a huge category with security implications. If the installation adds a network share or internet accessible directory, is the user being told this up-front? Take as an example the .NET framework installation. Is the user asked/told if I want ASP.NET installed to the IIS server? The answer is no. A mention of this or other types of global system changes in the documentation alone (especially since most documentation is installed with the product) is unacceptable. Note that most modern media player installations ask the user if he or she wants to associate file types with it. Installations were not always this nice to your system.

2. Not checking return codes. When running a command that is supposed to do something essential for a successful installation, does the installation check the return code and handle the error? A great example of this problem is found in many database installations, where the result of a SQL script is not checked. The application does not work after a reported "successful" installation, and it takes a tremendous amount of time and effort to determine why.

1. Not validating user input. The install allows the user to free-text something in where choices can not be predetermined (see pet-peeve #8); let's say a timeout value or something that is essential for the installation to succeed or the application to function. The user types in "10h" by accident. The install doesn't verify the input is a number or a number within a valid range. Installation fails (bad) or says it is successful, yet the application does not work (infinitely worse).

Aside from this list, a very useful document for installation (and application development)authors to follow is the "Designed for Microsoft Windows XP" Application Specifications. Many of the items listed there are also listed above.

Detecting .NET Framework Version (Update)

Aaron Stebner from Microsoft comes through with some source code (in C++) to determine the installed version of the .NET framework, including service packs. Great job, Aaron!

Too good to pass up

While I'd usually like to abstain from sharing personal or political viewpoints on a technical blog, this one was too funny. I take for granted everyone on the blogsphere knows about the Dan Rather / 60 Minutes II documents regarding Bush that appear highly likely to be forged. Well, there is a new product that can be used for document forgery. I can see Clippy saying, "So... It appears you are trying to forge a document..."

Detecting .NET Framework 1.1 SP1

Aaron Stebner from Microsoft blogged about how to determine if the 1.1 version of the .NET framework is installed. Unfortunately, that is just to detect if 1.1 is installed.

One method of checking that Aaron quoted is described by a Microsoft Knowledge Base article. This approach is impossible to perform programmaticly. He also gives a registry key to check for the 1.1 .NET framework, but is a bit unclear if this key is for 1.1 or 1.1 with the service pack.

Other methods of checking I uncovered are difficult to do in code in a manner that makes sense to work in the generic case. The confusion that led to Aaron's post in the first place are questions (and the answers/lack of answers) posed in newsgroups, like this one.

The .NET framework version checking problem reminds me of old InstallShield code. I can't recall if this was built into InstallShield or if it was a bad code sample, but lots of installs failed on Windows 2000 simply because the install code checked to see if you were NT 4.0 SP3 or higher. The detection code was not based on proper logic, but rather if OS version >= 4 and SP version < 3, do not install. What a headache (at least until Windows 2000 SP3 came out)!

Fortunately, Aaron is going to research getting a code sample to detect the old versions and hopefully Microsoft can provide a future-proof method to determine this in upcoming revisions.

Wrestling with the MSI setup editor in Visual Studio

Tim Anderson blogs about editing an MSI after it is built by Visual Studio .NET. There are various tips hidden in here about how to cope with duplicate assemblies being added to the MSI, marking a file so upgrades don't overwrite it, invoking the Custom Action editor, and MSI SQL.

For more detailed information related to editing an MSI after it is built, check my previous blog entry "MSI - Modifying the Installer Database at Runtime" and scroll down to Solution 1.

For more information as to how to launch a program after installation, see How do I use a custom action to launch an installed file at the end of the installation? from MSDN.

Tim also mentions as to why he stuck with using the Visual Studio MSI packager instead of a commercial editor. He gives 4 reasons, and I would add purity as a fifth. Regardless of the authoring tool he uses, his code will always work. Granted, pretty much any tool he uses will eliminate the need for his particular edits to the MSI, but the concept is still valid.

MSI - Modifying the Installer Database at Runtime

A recent question posted to the Microsoft.public.platformsdk.msi newsgroup:

I'm trying to prevent some NT services installation with a VBScript custom action (type 38) that removes the corresponding rows from the ServiceInstall table.

The poster shows some example code that does not seem to work. Let's look at some approaches to try to solve this problem.

Solution 1

Here's some theoretical JScript to run in a Custom Action that modifies the installer Database:

oDatabase = Session.Database;
oView = oDatabase.OpenView("select * from ServiceInstall");
oView.Execute();
//now our view object has records we can "fetch"
oRecord = oView.Fetch(); //get the first record from the query
while (oRecord != null) //make sure we are not at the end.
{
//TODO: Either handle the logic on what to remove based
// on the oRecord here or in the query. Assumes all
// records matching the query are removed.
oView.Modify(6,oRecord); //6 = msiViewModifyDelete
//get the next record from the query
oRecord = oView.Fetch();
}
oView.Close(); //close the view
oDatabase.Commit(); //commit our changes

Looks pretty good, right? We are opening the database, fetching records, modifying the records, and committing the changes. Sad to say even though the API logic is correct, this won't solve the problem at hand! Why?

Custom actions can only add, modify, or remove temporary rows, columns, or tables from a database. Custom actions cannot modify persistent data in a database, such as data that is a part of the database stored on disk. (View.Modify documentation excerpt)

As an aside, you could substitute the following 2 lines for the first one and modify the msi (permanently) outside a CA:

oInstaller = new ActiveXObject("WindowsInstaller.Installer");
oDatabase = oInstaller.OpenDatabase("myInstall,msi",2);

Solution 2

So, with this restriction, how do we solve your problem? The answer lies (buried) in the View.Modify documentation. The action type msiViewModifyInsertTemporary is our friend. Instead of solving the problem by deleting the services you don't want to install, we can add the services you DO want to install at runtime! Remember to delete the lines from the ServiceInstall table if you use this method

Lets look at an example of this:

oDatabase = Session.Database;
oView = oDatabase.OpenView("select * from ServiceInstall");
oView.Execute();
oRecord = Installer.CreateRecord(13);

oRecord.StringData(1)="MyService";
oRecord.StringData(2)="MyServiceName";
oRecord.StringData(3)="MyServiceDisplayName";
//SERVICE_WIN32_OWN_PROCESS
oRecord.IntegerData(4)=16;
//TODO: continue for the rest of the columns in this table. See
// the ServiceInstall table docs for more information

//msiViewModifyInsertTemporary the record.
oView.Modify(7,oRecord);

oView.Close(); //NOTE: no need to call commit..

You do need to deal with the uninstall issue using another CA. That is not covered here, but the concept is similar.

As an aside, I use the above technique to adjust the ReserveCost table when I need to create a large database. If the DB exists, I do nothing, otherwise, I add the size of the database to the table for proper costing.

Solution 3

Although Solution 2 will solve this problem, why would you want to go through that complexity when there is a much better (and simpler) method built into the MSI engine? In the ServiceInstall table, there is a Foreign Key into the Component table. If the component is to be installed, so is the service.

Simply create a hidden feature, assign the components necessary for the service to that feature, and conditionally install (or not install) that feature.