Custom Action Tutorial Part III – What we did in Part II

This is the third in a series of articles about building Custom Actions. If you are coming in late to the party, check out Part I and Part II first. If I tried to explain what we were doing and why in the last installment of this tutorial, you would be even more lost now than you probably are. This article is an attempt to explain exactly what it was we did in Part II and why. Let’s start by going over each project in our solution.

The HelloWorld project

We needed something to install. This happens to be it. I could have picked notepad.exe or something, but it is nicer to see how Visual Studio could grab project outputs and put them into an MSI fairly easily.

The CustomAction project

This one is really the meat of what we are trying to learn. The other projects are sort of extraneous if you are using a tool other than Visual Studio to build your MSI. Microsoft did a pretty good job of explaining why you may want to use Custom Actions in the first place, so I’m not going to rehash that.

We created a Custom Action DLL. The reason I chose a DLL is because it is the most flexible means of tightly integrating a Custom Action with the installation engine. Additionally, there is not much you cannot do in a Win32 C++ DLL. An executable Custom Action cannot interact with the MSI engine. A VBScript or JScript action does have this capability, but if you are not careful, these are not bulletproof, and usually won't allow you to do much anyway.

We added a "Module Definition File" (.def) file to the project. This .def file is read by the linker and is used (for our purposes, anyway) to prevent mangling of function names (to be technically correct, specify the calling convention to be used). Mangling is entirely normal when a DLL is created without using the extern "c" function declaration – in fact, it is called by the more pleasant name "Decorating" the function names. If you run "Dependency Walker" (depends.exe in the Bin folder of the Platform SDK), and open a DLL in it, the listing of function exports appears. Pick FrameDyD.Dll from the same Bin directory and look at it. You will notice some ? and @ symbols around the names of functions. The .def file allows us to not create these mangled exports – you can verify this by opening the custom action dll we built in our project and see this for yourself. Ultimately, this makes the process less prone to error when entering data in the custom action table of the MSI. When adding the Custom Action via Visual Studio, it will handle the mangling automatically, so for this specific project this was not strictly necessary. If you are adding this DLL using another installation designer, you will thank me for not mangling the function names. Anyway, the .def file must contain the function names of any function you wish to make callable from outside the DLL. In our case, we wanted all four of the functions we added to be callable by the MSI engine, which is why they were added to the DEF file. There are tons of other ways to accomplish this goal

The function signatures we added to the CustomAction.cpp file are based on the requirements of the DLL Custom Action type. We return an unsigned integer (UINT) and accept a single parameter that is the handle to the installation (MSIHANDLE). We must use this handle to interact with the MSI Engine. The UINT return type indicates our status to the MSI engine.

The file "msi.lib" is the link library for the MSI engine. All custom action DLL’s must link to this file in order to call any MSI functions. The instructions in Part I explained an alternate way to inform the linker what to link with. One way is not necessarily better or worse than the other. The functions defined in the header files msi.h and msiquery.h (that we included in the stdafx.h file) all require linking to this library.

We made additional solution configurations to give us a UNICODE project type. While this is not intended to be a primer on UNICODE, all NT kernel OS’s are UNICODE under the hood. Windows 95 through ME are not UNICODE, although they can support some UNICODE extensions with the Unicode Library. For dll's and executables that only will run under Windows NT, 2000, XP, or 2003, the only option you should use is UNICODE. Therefore, if you are targeting only NT based systems, build and use the UNICODE projects. If you are supporting Windows 9x based machines use the non Unicode versions. To make the Unicode transition easier, Microsoft provided the tchar.h header that we include in the stdafx.h file that deals with strings based on compiler settings (actually preprocessor macros), so you may write code that will compile and work as native UNICODE or ANSI based on compiler settings. The TEXT() macro used in the "Hello MSI!" exercise is an example of one such tchar.h macro. Future Custom Action examples presented here will use the tchar.h macros when the compatibility of the Custom Action spans both Windows 9x and NT based platforms. Actions targeting Windows NT based systems will only compile in UNICODE, and will not use the tchar.h macros.

The final thing we added to the CustomAction project was the header include for strsafe.h. This header file describes the safe string functions intended to replace the C/C++ standard library string functions as well as specific Windows string handling implementations. These functions always start with the word "String" followed by the type of count provided as a parameter to the function, either the "Cb" which is a count of bytes, or "Cch" which is a count of characters. Remember that Unicode implementations use 2 bytes per character as opposed to the ANSI 1 byte per character – so the Cb and Cch designations are important. The last important fact about this header is by default it will deprecate the unsafe string handling functions – if you use sprintf(), you will get a compiler warning.

The Setup1 Project

Hopefully we are pretty clear on the basics for the first part of this one – an EXE is generated, Visual Studio generates a basic MSI and adds entries in the Feature, Component, File, Shortcut, and other tables to install the HelloWorld exe and the shortcut for it.

When we added the Custom Actions, we did a few things that affected the compiled MSI. Use Orca (which you installed in Part I) to view the generated MSI and follow along - try to remember back to the Custom Action theory presented in Part I:

1. Added a new Component and File table entry that installs the CustomAction.dll file.
2. Added four entries to the CustomAction table, referencing the DLL that we installed with the product. Notice the Source column of the CustomAction table is a reference to the key of File table for our Custom Action DLL. The CustomAction table's Target column is the DLL entry point (remember this is nothing more magical than the function names in our .def file). The action name is just a randomly generated GUID. The type column starts its life as a Type 17 which indicates msidbCustomActionTypeDll + msidbCustomActionTypeSourceFile, and to this number is added the following:

   Action Type	Decimal Value	Description
   Uninstall	1041	= 17 + msidbCustomActionTypeInScript
   Install	1041	= 17 + msidbCustomActionTypeInScript
   Rollback	1297	= 17 + msidbCustomActionTypeInScript + msidbCustomActionTypeRollback
   Commit	1553	= 17 + msidbCustomActionTypeInScript + msidbCustomActionTypeCommit

3. Looking at the InstallExecuteSequence table, note that all but the uninstall custom action is scheduled before the RegisterUser Standard action, and the uninstall custom action is scheduled before the UnpublishComponents action. Note that these actions are conditionalized on the action state of the component, denoted by the dollar-sign - you can see this clearly in the InstallExecuteSequence table. In a nutshell, this condition will cause the custom actions to run only when the component it is tied to (the Custom Action DLL) is being added or removed from the system.

Visual Studio does not give us a bunch of options when dealing with Custom Actions, so we are pretty much stuck with what they offer – which is the ability to add an additional condition (other than the component install state), stuff what goes into the CustomActionData property fairly easy, and sequence the custom actions relative to each other. You cannot change sequencing of the custom action (like making it occur relative to another standard action other than RegisterUser or UnpublishComponents, embedding the custom action in the binary table, or add immediate custom actions. Other authoring tools offer the ability to be tons more flexible, but we are stuck with these limitations for now.

That concludes Part III of the Custom Action Tutorial. In the next part, we will learn how to debug the Custom Action.

Custom Action Tutorial Part II – Creating the Project

[Update 3/10/2009: Updated download link. Thanks for hosting, Aaron!]

In Part I we covered the dry material - what custom actions are, and how to use them. In this part of the tutorial, we will configure the development environment and create a baseline project to start from when developing future Custom Actions. Please make sure your system is configured as described in the first part before continuing.

I am providing a download link for the final project described in this part for convenience (see the end of this post for the link), but I recommend first time custom action developers follow the steps so they understand what is going on. For my more experienced readers, please bear with the simplicity here, but review the project settings before downloading or using the template in future parts.

Create a Basic Installation Project

To create our solution, open Visual Studio 2003. We are going to create two projects to start with – one a Win32 Project called "HelloWorld", and a deployment project that will create an MSI to install it. Go to the File->New->Project menu. Select the Visual C++ Projects->Win32->Win32 Project. Call the name of the project "Hello World" and place it into a directory such as C:\Projects\CATutorial. Select Finish in the Wizard dialog. When the IDE initializes, you should see the Solution Explorer window – Right-click on the Solution 'HelloWorld' line at the top of this box and select Add->New Project. Find the Setup and Deployment Projects category and select Setup Project, and select the Setup Project Wizard – you can accept the default name of Setup1. Hit next, and accept the defaults for page 2. On page 3, check the Primary Output from HelloWorld option and hit Finish. When the project opens, select the Users Programs Menu and right-click in the pane to the right. Select Create New Shortcut. Double-click Application Folder and double-click the Primary Output from HelloWorld (Active). Rename the shortcut if you wish.

Now we have both an EXE and an MSI project that installs the EXE plus a shortcut to it. At this point, we can build and test our MSI by right-clicking the Setup1 project and selecting Build. Next, right-click the project and select Install. You will be walked through the standard MSI project dialog sequence – just accept the defaults and finish the install. A shortcut is installed to Program Files in the Start Menu, which will open a window that pretty much does nothing. If you get this far, you are on the right track.

Add a DLL Project

To make a Custom Action DLL, we want to now add a new project – a Win32 Project that is a DLL. Follow the add project instructions for "HelloWorld" above, but call it "CustomAction" and in the Wizard, select the Application Settings and pick DLL for the Application Type. Press Finish. We now need to add some items to this project – please follow this closely.

1. Right-click on the CustomAction project and select Add->New Item. Find the Module-Definition File (.def) and select it. Call the file CustomAction and press Open. We'll add some content to it in a later step.
2. Right-click on the CustomAction project and select properties.
3. In the Configuration dropdown, select All Configurations
4. Expand the Linker folder, and select input. In the Additional Dependencies edit box, enter msi.lib and hit Apply. NOTE: As an alternate to this and the previous 2 steps, you can use the following directive in the stdafx.h or any other source file to tell the linker to link with msi.lib: #pragma comment(lib, "msi.lib")
5. In the upper right hand corner of the Property Pages dialog box, click Configuration Manager.
6. Under the Active Solution Configuration dropdown, select New. Enter the name Release Unicode, select Release from the Copy Settings from dropdown, and leave the checkbox checked. Press OK.
7. Repeat the previous step, calling the Configuration Debug Unicode and select Debug under the Copy Settings From dropdown. Close the Configuration Manager.
8. We are now back at the CustomAction property pages. Select Debug Unicode from the Configuration dropdown. Under the General Configuration Properties, find the Character Set setting and select Use Unicode Character Set. Press Apply.
9. Repeat the previous step for Release Unicode. Press OK to close the CustomAction property pages.
10. Repeat the previous two steps for the "HelloWorld" project by opening the property pages for the HelloWorld project.
11. In the CustomAction project, under the "Header Files" folder, double-click the file stdafx.h file to edit it. Under the // TODO: line, add the following lines:
    #include <tchar.h>
    #include <msi.h>
    #include <msiquery.h>
    #include <strsafe.h>
12. It is always a good idea to add versioning information to all the compiler outputs you build, so right-click on the CustomAction project and select Add->Resource. Pick Version from the list by double clicking on it. Edit the information here if you so desire.
13. In the CustomAction.cpp file, add four functions based on the following template named Install, Commit, Rollback, and Uninstall – function names are case sensitive.
    extern "C" UINT __stdcall Install(MSIHANDLE hInstall)
    {
    return ERROR_SUCCESS;
    }
14. Edit the CustomAction.def file and make it look like this:
    LIBRARY CustomAction
    EXPORTS
    Install
    Commit
    Rollback
    Uninstall

As a final step, we want to add the Custom Action DLL to the setup project. Right-click on Setup1 and select View->Custom Actions. Right-click on Custom Actions in the pane that opens up and select Add Custom Action. Double-click on Application Folder, and then click Add Output. Select CustomAction from the Project dropdown and accept the default Primary Output and Active Configuration settings. Click OK, then click OK on the Select Item in Project dialog. You will notice that four custom actions are added, and in viewing the properties for each of them the EntryPoint corresponds to the function names we created in the CustomAction.cpp file.

At this point, lets right-click on the Setup1 project and build it. You will notice that the file "msi.dll" is detected as a dependency for the Setup1 project – as a matter of fact, it will create a warning during compilation. Right-click on the msi.dll file under dependencies and check the Exclude option. Rebuilding the Setup1 project should now generate no warnings.

If you have not uninstalled the setup project from the earlier test, do so now, and run the Setup1 installation. You should see no difference from the previous time you ran the setup. If you get errors, you may want to download the zip file that contains the projects and steps we have performed up to here, and use WinDiff from the Bin directory of the Platform SDK to see if you can figure out the issue by comparing the directories of my project against yours.

Save your workspace and exit Visual Studio. Create a .zip file of the project directories – if you used my directory structure, you want to compress the "C:\Projects\CATutorial" directory. We will use this as a template for any future C++ custom action tutorials.

Initially, I was going to end this part here, but I thought people would be cursing me, saying that this whole exercise was pointless and no fun! After all this work, we created a Custom Action that did absolutely nothing! Let’s rectify that! Open your solution and replace the Install function in the CustomAction.cpp file with the following code (Don't worry if you don't understand it, we will cover this later):

extern "C" UINT __stdcall Install(MSIHANDLE hInstall)
{
PMSIHANDLE hRecord = MsiCreateRecord(0);
MsiRecordSetString(hRecord, 0, TEXT("Hello MSI!"));
MsiProcessMessage(hInstall, INSTALLMESSAGE(INSTALLMESSAGE_USER + MB_OK), hRecord);
return ERROR_SUCCESS;
}

When you rebuild the project and install it, you will see a "Hello MSI!" message when the Install Custom action runs.

You can download the Visual Studio 2003 Project Template here:

(Special thanks to Aaron Stebner for hosting this for me). The next installment in this series will explain what we did (and why we did it)!

Custom Action Tutorial Part I – Custom Action Types and Sequences

This is the first part of a multi-part series on Custom Actions in the MSI world. The articles are designed to be read in order, as each one will build from knowledge gained in the previous ones. To give you a short roadmap, the series will start with some dry and boring theory, and the subsequent articles will start us down a path of writing Custom Actions in unmanaged C++, C#, and script.

Custom Actions are used when the power of the Windows Installer engine's Standard Actions are not enough to accomplish a given installation task. For instance, installing or controlling a service is trivial using MSI Standard Actions, but creating a virtual directory in IIS is not. To create that virtual directory, you will need to write a custom action. Other examples of installation tasks that require Custom Actions can be found here.

Before beginning, I'd like to make sure you have the proper environment to work under. I'm going to use Microsoft's Visual Studio 2003 as the environment of choice for this series. The writing level is intended to be a walkthrough for someone who may be academically familiar with C/C++. I am going to handwave much of the Win32 enumerated types and definitions - the reader who is completely unfamiliar with these topics can find additional resources in books, the MSDN library, or around the web.

This part covers some necessary background on the different Custom Action types and Custom Action scheduling. Mostly this is dry and boring theoretical stuff, but essential to understand. Hopefully, I summarized it fairly well and won't cause anyone to drift off to sleep during their lunch hour.

Setting up your environment

For this first part, you only need the Platform SDK and a free MSI editor called Orca (distributed with the Platform SDK) installed. Future parts will require the following setup, so prepare yourself by setting this up. The development environment should be set up as follows: Install Visual Studio 2003, the current MSDN Library, and the most current Platform SDK. The Platform SDK contains new and updated header files and link libraries to support the latest OS platforms, components, and service packs. The installation of the Platform SDK installs a shortcut Microsoft Platform SDK in the Program Files menu. Open this and find the option to register the platform SDK directories with Visual Studio and run it. Finally, in the installation directory for the SDK, there is a directory "bin" and inside this you will find orca.msi. I recommend installing Orca, as my instructions are based on this. You may also substitute the MSI editor of your choice, just be aware of the differences.

Let's get started

We are going to use Orca to investigate the Orca.msi file. So, find the Orca.msi file, right-click on it, and select Edit With Orca, and play along.

The Two "categories" of Custom Actions

There are only two major categories of custom actions. The category of a custom action is indicated by a number in the type column of the CustomAction table (You should be finding this table in Orca as you read along). This number is actually a bitmask - for folks new to programming, if you looked at the number in binary form, the presence of a 1 or 0 in a particular location is how all the flags and options are interpreted by the MSI engine. I will give the hex (or Hexidecimal) representations of these flags throughout (all hex digits begin with a 0x to indicate they are hex). You can use the Windows "calc" program in the scientific view to enter the hex and convert it to binary to see which flag toggles which bit. Using hex (or even binary) is perhaps the best way to reverse engineer the decimal value from the table into the flags it represents. The two major categories of custom actions are:

• Deferred - This is a modifier of (in hex) 0x400 or 0x4000 in the Type column of the CustomAction table, which corresponds to what MSDN calls (and you do not need to understand what these are yet) msidbCustomActionTypeInScript or msidbCustomActionTypeTSAware, respectively. What is important is Deferred actions are actions that modify the state of the system in some way. There are also some notable restrictions on deferred actions - you cannot change the value of a property, you can only read the value of a single property, and you cannot interact with the MSI tables. These can only be sequenced between the InstallInitialize and InstallFinalize actions in the Sequence table, lest you encounter the dreaded 2762 error "Cannot write script record. Transaction not started." error. You will find out the reasons behind this error later in this article. Additionally, there are options to specify a special kind of deferred action of "Rollback" or "Commit." These will also be covered later in this article.
• Immediate - This is a modifier of zero in the Type column of the CustomAction table - in other words it is any entry in the Type column that does not contain the bitmasks described in the Deferred section above. Immediate actions are used for UI tasks, evaluating the state of the system, or preparing data or the system itself for later modification from a Deferred action. Immediate actions should never modify the state of the system.

The Orca.msi I am looking at has two different custom action types - 0x23 and 0x33. Neither of these numbers has the 0x400 or 0x4000 bits set, so this must mean they are Immediate Actions.

The Type column of the CustomAction table

Custom Actions have a Type - this is the base number (or bitmask) that gets entered in the Type column. This Type column tells the MSI engine both what they do and how to interpret the Source and Target columns of the CustomAction table. The major Types are:

Action	Hex Code	Description / Notes
Call a DLL function	0x01, 0x11	The DLL function that is called must be specifically written for the MSI engine.
Run an EXE	0x02, 0x12, 0x22, 0x32	Any EXE will run. By default, if the EXE returns a code other than zero, the action will fail. There is a flag available to turn off this behavior.
Abort the Installation	0x13	Aborts installation and displays a Formatted message from the target column.
Set a directory	0x23	Uses formatted text from the Target column to set a Directory.
Set a property	0x33	Uses formatted text from the Target column to set a Property.
Run some Jscript	0x05,0x15,0x25,0x35	Bear in mind the normally present WScript object is not available in these scripts.
Run some VBScript	0x06,0x16,0x26,0x36	Bear in mind the normally present WScript object is not available in these scripts.
Run a nested installation	0x07, 0x17, 0x27	Nested installations are evil. Don't do it. Use a setup launcher or AppSearch to enforce setup order.

Hopefully you see the pattern here - all the EXE actions contain 0x2, all the JScript ones contain a 0x5, etc.

The Source Column of the CustomAction Table

The second hex digit from the right in the CustomAction Type column tells us how to interpret the Source column. This is a bit more of a stretch then above, but you should see the trend:

Hex Code	Description / Notes
0x00	The Source column is a key into the Binary Table, where the the file needed to run this action is stored.
0x10	The file needed for this action is installed as part of this installation, the Source column is the key into the File table. This fact places restrictions on where this type of action can be placed in the installation sequence. In the case of the nested installation, the Source column points to the msi path inside this msi's source tree. For the abort action, the source column is ignored.
0x20	Source column is the key into the Directory table for setting the directory or indicating the EXE's working directory,Source column is the ProductCode for the nested installations,Source is null for the VBScript and Jscript actions. Note that you cannot return anything other than a success code with actions of this type. Actual script is stored in the Target column of the CustomAction table.
0x30	Source is the Property name where the value of the property will be set by the set property action, Source column contains the script code for script actions, or for the EXE action the Source column is the property that contains the path of the EXE to run.

The Target column of the CustomAction table

The Target column is dependent on the major type of the CustomAction, reading the MSDN Summary List of All Custom Action Types will explain this column in more appropriate detail. Looking back to the Orca.msi file - the actions listed translate to the following:

• 0x23 - Set the directory listed in the Source column to the Formatted Text in the Target column.
• 0x33 - Set the property listed in the Source column to the Formatted Text in the Target column.

How and When is a Custom Action run?
Now that we know a Custom Action is either Immediate or Deferred, and can be or do just about anything, we need to answer the question "How does a Custom Action get fired off?" Entering a Custom Action into the CustomAction table really only makes it available to be called - so just adding a line in the CustomAction table effectively does nothing. There are three methods of firing off a Custom Action:

• In the sequence tables. This is probably the most common way of running a custom action, and the focal point of our discussion for the remainder of this article series.
  
• In the ControlEvent tables. Essentially this is the method to launch a Custom Action in response to a UI event such as a button click. On Windows Server 2003 or better, custom actions run this way cannot send messages with MsiProcessMessage() calls, however Session.Message() calls from the automation interface work just fine (technically, prior to Windows Server 2003 neither message API officially works). Obviously, the installation must be running in full UI mode to fire these off. The Custom Action type launched by this method must be an Immediate type - remember we should never alter system state in the UI phase. I'm not going to cover how to run an Immediate Action from a dialog here using ControlEvent, perhaps in a later article.
  
• By a call to Session.DoAction() or MsiDoAction(). This is a way of calling one Custom Action from another. Although this won't make sense until it is explained later on, you can not call a Deferred Custom Action in this manner unless it is called while the "installation script" is being written - but it should be OK to call it from a custom action scheduled between InstallInitialize and InstallFinalize. More on the "installation script" concept later.

The Sequence tables

Before we can discuss the primary way Custom Actions are launched, you need a better understanding on how MSI sequences work. We will get into this in more detail later in this article so I am handwaving some important details here for the sake of getting a basic understanding. The typical installation is kicked off by double-clicking the MSI. This causes the installation engine to begin processing the InstallUISequence table (The processing of this table is skipped entirely if the UI mode is set to "basic" or "no UI" - both Full and Reduced UI modes actually do process this table). There are other sequences such as the AdminUISequence and AdvtUISequence that all follow the same logic for administrative and advertised installations. Hopefully you still have Orca open, so look at the tables as I describe them and follow along - sort the table by ascending sequence number.

The InstallUISequence table is processed in the order identified by the Sequence column (starting at sequence number 1), executing each action listed(Standard, Custom, and Dialog), as long as the Condition column for that action evaluates to true or is null (empty). The negative values in this table are "jumped" to on termination of the setup based on the result code of the installation. Once the "ExecuteAction" action is encountered, the InstallExecuteSequence is run. When the InstallExecuteSequence is complete, control will return to process the remaining UISequence table or it will jump to the corresponding negatively numbered sequence based on the result code of the installation process. Every action in the *UISequence tables are run in the currently logged on user's process for security reasons, and this is known throughout MSDN's documentation as the "client" portion of the installation.

The InstallExecuteSequence is where things get interesting, and differs based on platform.

• On Windows NT based systems, this table is evaluated and run in a separate process from that of the User Interface. See my earlier article on Properties for information as to what is passed from the "client" (UI portion) to the "server" or "service" portion of the installation. To recap, the admin user has two processes owned by him/her, one is the UI sequence processing, and the second is the Execute sequence processing. The non admin user installing with elevated privlidges can have three processes - the same two as above, plus a third process, owned by the System, which executes the deferred
  msidbCustomActionTypeNoImpersonate actions.
• On Windows 9x based systems, both the *UISequence and *ExecuteSequence and all actions contained therein are run in the same process.

Actions in the *ExecuteSequence should only use UI interactions that use the Session.Message() or MsiProcessMessage() API's and not reference the Dialog table or contain UI's (such as MessageBoxes, status bars, shell progress indicators, dialogs from 3rd party libraries, etc.) in order to respect the UILevel wishes of the user.

Now or Later

Let's continue the discussion by distinguishing the Immediate vs. Deferred custom action categories a bit more. An immediate action is one that is "run" or "executed" immediately when the action is encountered in the sequence. A Deferred action is one that, when encountered in one of the Sequence Tables, is written to a sequential "installation script" with some metadata - you can think of this as a "To-Do list". Wherever you see the words "script" or "written to the script" in MSDN documentation, this is what I am referring to. More on deferred actions and scripts later - let's get the Immediate Custom Actions out of the way first.

Immediate actions have a method of assuring they will only be executed a set number of times - this is perhaps the most confusing page of documentation in the MSDN library. The Custom Action Type msidbCustomActionTypeFirstSequence will cause the action to not run in the Execute sequence if it already ran in the User Interface sequence. The msidbCustomActionTypeOncePerProcess flag is similar to the msidbCustomActionTypeFirstSequence flag, with the exception that it will run an action again in the Execute sequence if the Execute Sequence is being run in a different process than the User Interface process (In English - it will run in both sequences on NT machines, and only once on 9x based machines). The msidbCustomActionTypeClientRepeat flag means if you run the MSI in silent mode (and therefore don't have the UI sequence table processed) , the Custom Action wouldn't execute, but if you ran it in full UI mode it would execute. The lack of any of these flags causes the action to be run always. Remember that Immediate Actions should not modify the state of the system in any way. Deferred actions cannot have the properties described in this paragraph. Deferred Actions are "recorded" in what is called the "installation script" before anything is actually run/executed. The script is created in the *ExecuteSequence when the InstallInitialize action is encountered, therefore deferred actions can only be sequenced after this action. The only time an item is written to the script is if the Condition is true at the time it is to be written to the script (as indicated in the Condition column of the Sequence table). In reality, there are two scripts being generated, the "installation script" and the "rollback script". I'm going to treat it as if there is only one script for simplified understanding.

Deferred Actions and the script

Think of this example: The MSI Engine is going through each of the actions in the InstallExecuteSequence in order of the sequence numbers, when - blammo - it hits the InstallInitialize action. At this point, it creates a data structure known as the "script" and starts recording all actions that meet the condition specified in the Condition column and are marked as msidbCustomActionTypeInScript along with some magical metadata (In other words it only records deferred actions in the script). If an immediate action with a proper condition (that also is compliant with the multiple execution flag setting) is encountered, it runs immediately. Otherwise, it keeps adding stuff to the script until it hits an InstallExecute, InstallExecuteAgain, or InstallFinalize action is encountered. Ignoring InstallExecute and InstallExecuteAgain for now, when InstallFinalize is encountered, it completes writing to the script and seals it from further additions. Then it begins processing the script by executing the actions, top down. The details of how the list is processed and the effect of InstallExecute and InstallExecuteAgain is covered in a bit.

So, what exactly is this magical metadata mentioned above? We already know that the condition column was evaluated before the decision was made to write the action to the "script" - if the condition was false, the task was not written. As near as I can figure, this metadata holds three major things:

• The value of a property that having the same name as the CustomAction - the so-called CustomActionData property you have undoubtably heard so much about. Additionally, the ProductCode and UserSID properties are available in a Deferred action.
• How to Run it flags (derived from the Type column of the CustomAction table)
• The type of action (DLL, EXE, Script, etc.) and the path to the CustomAction code.
  
• Impersonation type - The flag msidbCustomActionTypeNoImpersonate will cause the action to run in the system context vs. the otherwise usual currently logged-on user context. The exception to this rule is a server with Terminal Services which normally runs actions in the system context unless the msidbCustomActionTypeTSAware flag is set (flag valid on Windows 2003 Terminal Server only).
  
• msidbCustomActionTypeAsync indicates that once the action is kicked off there is no need for the MSI engine to wait for it to complete before executing the next action. However, the MSI engine does want to know the return code, and as such will wait around at the end of the sequence until it finally returns if the action takes that long. All actions are synchronous unless this flag is set. Only EXE type custom actions can also combine the msidbCustomActionTypeContinue flag so that the installation can exit even while the Custom Action EXE is still running asynchronously.
  
• msidbCustomActionTypeContinue indicates that the exit code of the custom action is ignored - therefore the sequence will progress even if the return code indicates failure.


• When to run it flags indicates three possible things (again derived from the Type column of the CustomAction table). As briefly mentioned earlier there are actually two different scripts written - Commit and Rollback actions are both written to a separate "Rollback Script", but ignore this for now.
• "Normal Deferred" assuming nothing has failed and the list is being processed top to bottom - msidbCustomActionTypeTSAware or msidbCustomActionTypeInScript
  options without msidbCustomActionTypeRollback or msidbCustomActionTypeCommit flags
• "Commit" - Run after the entire (sealed) script is processed once successfully (After InstallFinalize is encountered and the complete script has been processed once successfully). Actions in this category have themsidbCustomActionTypeTSAware or msidbCustomActionTypeInScript combined with msidbCustomActionTypeCommit flags in the Type column. This is only written if Rollback is not disabled - more on this later..
  
• "Rollback" - if another deferred action failed, this will be run. This is only written for the msidbCustomActionTypeTSAware or msidbCustomActionTypeInScript
  when combined with msidbCustomActionTypeRollback actions, and only if Rollback is not disabled - more on this later..

How the Script is Processed

To recap, we have a "script" containing deferred actions along with some metadata that we started collecting when InstallInitialize was encountered in the sequence. We can potentially start processing the list before it is completely written (assuming a InstallExecute or InstallExecuteAgain was encountered prior to InstallFinalize in the sequence tables).

Processing always begins top-down, and there are up to three "phases" in the script processing based on the "when to run it" metadata described above.

First, only "normal deferred" actions are processed. Each deferred action is executed only once - if an InstallExecute or InstallExecuteAgain action was encountered in the sequence tables causing a portion of the script to be executed prior to the script being completely written, "normal deferred" actions already executed are skipped.

If an error is thrown during the execution of a "normal deferred" action, the processing of the list reverses and starts moving backwards from where it currently is processing, executing the "On Rollback" actions in the list. Rollback can be "turned off" at a system level, if this is the case, Rollback actions will not be written to the script, and will not be executed.

Assuming all "normal deferred" actions are successful, processing of the script begins executing at the top again, this time executing the "On Commit" actions. Retuening a failure from a Commit Custom Action will cause any previously written Rollback actions to be executed. Because this causes some interesting and practically impossible to test scenarios, it is important to write solid Commit actions that do not fail or also specify the msidbCustomActionTypeContinue flag in the CustomAction table. Rollback can be "turned off" at a system level, if this is the case, Commit actions will also not be written to the script, and will not be executed.

Dealing with Rollback and Commit

If you are familiar with database concepts, Windows Installer is transactional. In other words, a failure to install a MSI package should leave the system in the exact state it was prior to beginning the installation. As such, Rollback and Commit actions are a part of the lexicon of Windows Installer, and understanding this when writing a Custom Action is essential. Also essential is understanding that Rollback can be disabled on a system level, thus, Rollback and Commit actions are not executed.

To adhere to the Windows Installer "Best Practices," all changes that are made to a system are in a "deferred" action - as an example, the "InstallFiles" standard action is actually a deferred action under the hood. Although not apparent in this manner (nor is apparrant by looking at the Sequence tables), the InstallFiles "normal deferred" action creates a backup of all the files it replaces if Rollback is enabled. This way, if an error occurs, all the replaced files can be restored to their previous state if an error occurs in a later deferred action through the Rollback process. The Commit actions are only run after all the "normal deferred" actions are run successfully - and the MSI engine uses the Commit action to delete the Rollback files.

Because of their nature, Rollback and Commit actions cannot be asynchronous.

A "normal deferred" Custom Action should only save rollback information if the RollbackDisabled property is not set. If it is set, Rollback and Commit actions will not be run, and this will cause your saved rollback data to remain after the installation is completed. You will need to pass this data into your "normal deferred" CustomActionData property in order to check if you should save rollback information.

Other Considerations

Inside a CustomAction, you may need to evaluate if a patch or repair is occurring. Since deferred CustomActions do not have the ability to check properties (other than the few properties mentioned earlier), you need to use an Immediate action to check (and pass on to the deferred action via CustomActionData) the value(s) of REINSTALL, PATCH, and/or MsiPatchRemovalList properties. For more details, see the Windows Installer Team Blog's comments on the subject or Heath Stewart's post.

To determine if you should save Rollback information in your deferred action, you will need to do the same technique as above to pass the value of the RollbackDisabled property. Other properties of interest for deferred CustomActions (depending on what they do) is ALLUSERS, and possibly UILevel.

To handle user requests to cancel the installation in any custom action, make sure that you check the return value from MsiProcessMessage() (or Session.Message()) calls to handle the IDCANCEL return value. I will be showing examples of handling progress messages in future posts on this topic. In the meantime check out the Windows Installer Team blog entry on the subject.

Another tricky concept is scheduling a reboot, if required, from a deferred Custom Action. Normally, you can't! To work around this oversight you have several options. The one I use is creating a file on the filesystem that the user running the installation can have delete access to. This is where the ProductCode and UserSID properties available to a CustomAction come in handy. Find the temp directory of the user using their SID and write a file with the name "ProductCode_rebootme" there. In an immediate action scheduled after InstallFinalize, check for the existance of this file and if present, delete it and call MsiSetMode() to indicate a reboot is required.

MSDN provides a few pointers on security here and here. In my opinion, it is extremely difficult to author a "data secure" MSI. If you need to pass internally sensitive data to a deferred custom action, use encryption on the CustomActionData in addition to adding sensitive properities (and the name of the deferred custom action) to the MsiHiddenProperties and using the msidbCustomActionTypeHideTarget CustomAction type flag. It is possible to digitally sign an installer package and its cabinet files. It is trivial to circumvent these measures if someone wanted to - and in a typical administrative installation, the msi signing is removed anyway. Think about data security in an MSI as glass in a jewelry store - it only keeps the honest people honest.

If a Custom Action needs to use disk space (to install a database, for example), you need to author (or modify at run time) the ReserveCost table. This is keyed on the installation state of a component.

The use of COM in a CustomAction is fine - when a dll-based Custom Action runs, it is created on its own thread. There is some conflicting documentation on this, however. MSDN claims CoInitialize is called in a per-machine installation and is not called in a per-user installation. A post here (scroll through to the comments) indicates that CoInitialize and/or its brethren CoInitializeEx is not called on this thread. I like MSDN's advice to not quit if it is determined the thread is already COM initialized.

Terminal Server is special. Throughout this article I have mentioned the *TSAware flags. In a per-machine installation on a Terminal Server, actions not marked with this flag are run as LocalSystem, where on a non-terminal server system they will normally impersonate the calling user.

The way patching and upgrading is handled, and its effect on Custom Actions, is the subject of a future article in this series.

The Ten Cent Summary

Custom Actions can be very powerful, and can be run in a variety of different ways and at a variety of times. The important things to remember when deciding on how to design them are:

• Immediate actions are normally used to "set up" deferred actions (by stuffing CustomActionData) or to simply evaluate the state of a machine. Immediate actions are NEVER to be used to alter the state of a machine.
• Deferred actions come in three different flavors, and where you have one, you should have all of them - the "normal deferred", rollback, and commit. The "normal deferred" should save undo information on the system somewhere if rollback is enabled. The rollback action will use this undo information to restore the initial state of the machine. The commit actions will clean up the rollback's undo information. Deferred actions can only be included in the execute sequences between InstallInitialize and InstallFinalize.
• Rollback only occurs if an error is encountered while processing deferred actions (including Commit actions) between InstallInitialize and InstallFinalize in the *Execute sequences.
• Because the conditions of deferred actions (including commit and rollback actions) are evaluated at the time the script is written, the same condition should be used for the sister actions.
• Actions should be written to run correctly regardless of the UI mode and respect the wishes of the user when it comes to UI levels.
• Never use a nested installation custom action type.
• Understand the flags and options for scheduling and running actions as presented above.

Part II of this series will cover writing our first Custom Action in C++!

Installing an Internet Explorer ActiveX Control Part II

In part I of this article, I expressed concern over the various methods of installing an ActiveX control normally distributed and/or updated in a CAB file via Internet Explorer. This follow-up post addresses some concerns of a commenter to that last post, and describes what my solution was.

The comment made by Troy was from the perspective of a repackaging solution. Troy is quite correct - using pre and post installation snapshots will grab the changes to the ModuleUsage subkey. However, this solution fails to achieve the other goals for this particular installation. We still have the issue of a later update coupled with the MSI repair (or self-repair) operation. A point I failed to mention but Troy unintentionally reminded me of is compatibility across versions of IE. With the type of software this is, it may have a shelf life that would include IE7. Are the changes made to the system under IE6 applicable to all versions of IE under all OS's? I don't know. Using the API to install the control alleviates most (but not all) of the potential future incompatibility concerns.

I especially like how he disliked option #3 - changing the IE security settings to assure a successful installation, calling the API, and resetting them. I laughed as I wrote this option, as I could not imagine anyone doing this. In the middle of last week, I came across the "Get In the Zone" section of an issue of "Web Team Talking." They actually suggest and provide code for doing just that - although in a different context for a slightly different reason. I happen to agree with the KISS directive - "Keep it Simple, Stupid" - but sometimes to assure a successful operation you need to handle the corner cases.

The last part of his comment - "[this is] why many enterprise customers choose to repackage installations from ISVs" is a completely different blog post. I cannot tell you how true this statement is - and it is usually caused by setup developers, either intentionally or unintentionally. I'll add this one to the list of future topics for sure.

For the sake of discussion we were using the Crystal Reports Web Viewer which can generically be replaced with any properly signed cab file that works in the normal deployment mechanism of IE using the OBJECT tag. I have not tried this with unsigned controls, but I tested the approach using Macromedia's Flash player. You can play along by visiting the Macromedia's Flash download page and getting the CLSID from the OBJECT tag in the page's source. I then downloaded the swflash.cab file from the codebase attribute of the object tag and stored it on my local disk. The example below is going to use this flash cab file and the flash Class ID. Additionally, I am going to assume the reader is familiar with COM, C++, and MFC. The code is a bit lengthy to post here, but using the steps below a reasonable Windows developer should be able to complete this exercise in an hour or two.

I chose MFC since we need to implement IBindStatusCallback. This is easily accomplished by deriving a class from CCmdTarget. Make sure the implementations of QueryInterface(), AddRef(), and Release() call the External...() base class implementations. GetPriority() and OnLowResource() can simply return E_NOTIMPL. All the other methods can return S_OK. Since this is an async callback we need to have an event handle member variable, initialized in the Constructor, and have the event set in the OnObjectAvailable() method. That's the essential plumbing for the callback class. The callbacks for download progress can be implemented for the progress bars in the MSI.

Before we continue, let's make sure we have all the data we need to install the control.

• The class id of the control is {D27CDB6E-AE6D-11cf-96B8-444553540000}. It needs to be in this format as a string (with the braces) so we can call CLSIDFromString to convert it to a CLSID type.
• We are going to use major and minor versions of 0xFFFFFFFF to assure it is always installed.
• Finally, we need a codebase - or place to "download" the cab file from. Since we are installing the cab file locally, the CA needs to be appropriately place in the sequence and we need to get the local path to the file. The path needs to be in appropriate format - for a local disk where the file is at C:\swflash.cab, we need to make it file://c:\\swflash.cab

The entry point code needs to hook up the plumbing - instantiate the IBindStatusCallback implementation class, call ExternalQueryInterface() on it to get the interface pointer. Then we need to create and register the binding context by calling CreateBindCtx() and RegisterBindStatusCallback().

The meat of what we are doing is a call to CoGetClassObjectFromURL() with the parameters described above. The two common returns from this call will be
MK_S_ASYNCHRONOUS, S_OK, or an error. If S_OK is returned, the object is already installed. If the async value is returned, we need to loop until the event in the callback class is set or some predefined timeout elapses. After the event is set, check to see that the object was created based on what was set in the OnObjectAvailable() method.

Based on testing, this method appears to work all of the time, regardless of the various IE security zone settings. This is likely due to using the file:// protocol. Remember that we are dealing with a signed control - with unsigned controls, your mileage may vary. This method was not tested in IE with Server 2003 in Enhanced Security Configuration, so all bets are off there as well.

If you wish to have the installation always grab the "latest" version of the code from a URL, you will need to implement the code to change and restore the IE security settings from the article linked to above and also implement the download callbacks to move the MSI progress bar. Be sure to alert the user that you will be temporarily altering the IE security settings, and make sure when handling a cancel button press from the MSI that you restore the settings. This particular corollary to the problem can be useful if you want to refresh the clients using a simple repair/maintainence/reinstall option after the server-side control was updated without having Bob the network admin do much extra work (or having to release client side patches).

Installing an Internet Explorer ActiveX Control

This post is different from my usual posts, mainly because I have no idea what I am talking about! Well, at least I am admitting it this time!

I am tasked with creating an installation for an intranet type application. I need to create a shortcut to a URL and install a client side ActiveX control. For the sake of argument lets make it a fairly common one, such as the Crystal ActiveX viewer control. Sounds pretty simple, eh?

Let's go over a typical usage scenario. Hopefully, when designing applications and their deployment packaging you have some made-up profiles of typical users. Lets make up a few for our purposes. "Bob the system admin," "Mary the power user," and "Joe the restricted user" should be good enough. We can be a bit more specific and cover XP SP2 users, users with severely restricted ActiveX Browser settings, and non-IE browser users as well, but lets ignore these for now.

Essentially, Joe has a problem. We can give him the URL to the web application or even create a shortcut and/or favorite for him. The problem is, the first time he tries to run a Crystal Report, he gets a security warning, and the Crystal ActiveX control does not install. Of course, it is because he has no permissions to install software, even if everything else we ignored above is in good shape. Incidentally, if we can't figure out that Joe's problem is that he is a limited user, you can try the CDLLogViewer application to find out why.

Bob and Mary never have issues with the system. In fact, if Bob and/or Mary perform the same steps as Joe everything works, and Joe logs in later on, even Joe can view and print reports. The problem is that Bob and Mary would never run reports on the system. Heck, Bob doesn't even launch the application. Bob may even deploy the install package using SMS or Group Policy.

In the corporate network world, pretty much everyone is a Joe. There are a few Privileged Mary's, and even less Bob's. To make all this work, Bob needs to install applications for the likes of Joe. This behavior is not only acceptable, but practically expected.

So how would Steve, our installation guru, make all this work? He would not only need it to work, but be practically bulletproof.

Option 1: The easiest approach. 99% of the time, this is synonymous with the wrong approach. We can extract the Crystal Reports ActiveX viewer cab file, read the .inf file to figure out what it is doing, add the files to a MSI project selecting the proper registration options, create the shortcut file, and be done with the project in under 20 minutes.

Why it doesn't work: Since Joe is not the only user of the machine (remember Mary and Bob?), any updates to the control will cause the MSI-based installation to screw up at some point. If the control is updated by Mary, and Bob uninstalls the application, the component/reference count then causes havoc with Mary by possibly removing the file on uninstall. Effectively, we have precluded the normal installation/updating/removal of the ActiveX Viewer by repackaging it in an MSI. More information on the hazards of this approach can be found on MSDN in the "Registry Details" page. Bad stuff.

Option 2: Install the shortcuts as in Option 1. Call the appropriate API's to install the CAB file from the web server. In this case, (after a bit of digging through MSDN I see this process is called "Internet Component Download" or ICD for short) we need to call CoGetClassObjectFromURL.

Why it doesn't work: Referring to the ICD link above, we assume the web server is up and running at installation time - #1 in the "Download and install process". We also assume that WinVerifyTrust returns a value that is compatible with the setting for ActiveX controls for the Zone in which the server we are pulling the cab file from resides - #2 in the "Download and install process". Last, but not least, we are assuming that good old Bob didn't muck with the CODEBASE setting of the Internet Search Path.

Option 3: Completely decimate the user's machine by querying the IE Security Settings, saving them, adding the server to a trusted zone, modifying the zone parameters to set the ActiveX settings appropriately, call CoGetClassObjectFromURL, after it succeeds, set everything back to the way it was.

This seems like a heck of a lot of work, but it appears to be the only way to bulletproof an installation of an ActiveX component. Plus, we ignored several issues. If the default browser was (or will be) Firefox or even Lynx, a shortcut to this URL will always fail to display the reports. To solve this problem, an EXE wrapper would need to be written to host the IE control to assure that IE is actually the browser engine used to access the page. Joel, the API war is far from over - it just made things more complicated!

Of course, I'll get to play around with these API's and see if installing the cab file from a file:// UNC path changes anything. I'll report back what I find. If anyone else has already "solved" this problem using an approach I haven't considered, please let me know. In all likelihood, I'll end up calling CoGetClassObjectFromURL, and if it fails, try to provide some meaningful feedback to the user as to why the installation can not succeed and let Bob worry about the corner cases. After all, he ultimately created them.

UPDATE: Please see Part II of this post!

Custom Action Tutorial coming soon to a blog near you...

Since I didn't get any responses for suggestions for a C++ custom action, I took a look at the newsgroups and found this post where the author wants to detect if an application is running and prompt the user, close the application, etc. Developing a custom action to handle this would be a good thing for the community, as it is a fairly common request.

This also meets the bill for covering almost every aspect of a robust custom action - user interaction, running in silent mode, custom table, validation, etc. The one thing it does not implicitly have is a rollback action. I will "fake" a rollback action for demonstration purposes.

I also decided on a multi-part series as the best format for this.

• Part one - describe the specifications and set up the development environment.
• Part two - create a "Hello MSI" custom action and insert it into a sample MSI.
• Part three - debugging the "Hello MSI" custom action.
• Part four - add tables and other required entries to the MSI.
• Part five - add code to read the tables.
• Part six - add code to close the application. I'll likely gloss over this pretty fast, as killing an application is not the true focus here.
• Part seven - testing the custom action.
• Part eight - creating documentation for other developers.
• Part eight - Creating validations for the action and tables.

In the process of developing this, I will loosely cover internationalization concerns, Unicode support, MSI Custom Action fundamentals, and more. I will "try" to follow the format of MSDN documentation in the process.

My intention is to complete the series first and post one new part every couple of days to keep the writing style similar and assure completeness. With my current workload, I'd guess part one would be published in late October.

Does anyone have specific feature requests for this Custom Action? Topics not mentioned here that should be covered?

Making Progress Bars behave nicely

After my previous blog entry about installation pet-peeves, including how writing progress bar code is hard, I figured I would try to help a bit by providing a simple sample for MSI authors. At the same time, a question related to progress bars appears on the newsgroups. Coincidence?

If any of you have some stylesheet stuff I can add to make posting/reading code snippets on a blog less painful for both you and me, please send it in.

The usage in a Custom Action is pretty simple: you parse CustomActionData if you are running deferred and determine how many things you have to do. Then call setupProgress("MyCA","Transmorgifying files",10); if there are 10 things to do. When performing each of the 10 things, shoot off a call to tickProgress("this filename"); and the bar should behave nicely. It's pretty easy to convert to the Custom Action language of your choice. Most of the information needed to create this sample is in the Session.Message() documentation.

var msiMessageTypeProgress = 0x0A000000;
var msiMessageTypeActionStart = 0x08000000;
var msiMessageTypeActionData = 0x09000000;

function setupProgress(ActionName, ActionDescription, numTicksExpected)
{
//set the UI Status ACTIONSTART
var statusRecordObj = Installer.CreateRecord(3);
statusRecordObj.StringData(1) = ActionName;
statusRecordObj.StringData(2) = ActionDescription;
//below line defines a template for display on each message tick
statusRecordObj.StringData(3) = "[1]";
Session.Message(msiMessageTypeActionStart, statusRecordObj);

//resets the progress bar.
var progressRecordObj = Installer.CreateRecord(4);
progressRecordObj.IntegerData(1) = 0; //0 = reset bar
progressRecordObj.IntegerData(2) = numTicksExpected+1;
progressRecordObj.IntegerData(3) = 0; //0 = left-to-right, 1= R-to-L
progressRecordObj.IntegerData(4) = 0; //0 = calc time remaining
Session.Message(msiMessageTypeProgress, progressRecordObj);

//tell the progress bar to increment for each ActionData message
progressRecordObj.IntegerData(1) = 1; //Action Info
progressRecordObj.IntegerData(2) = 1; //numTicks each ActionData
progressRecordObj.IntegerData(3) = 1; //increment bar for each ActionData
Session.Message(msiMessageTypeProgress, progressRecordObj);
}

function tickProgress(message)
{
//The ActionData record follows the template defined in the msiMessageTypeActionStart
//item #3. Refer to the MSI documentation related to templates. If you add more fields
//to the template, make sure to add the additional fields here as well.
var progressObject = Installer.CreateRecord(1);
progressObject.StringData(1) = message;
Session.Message(msiMessageTypeActionData, progressObject);
}

MSI - Modifying the Installer Database at Runtime

A recent question posted to the Microsoft.public.platformsdk.msi newsgroup:

I'm trying to prevent some NT services installation with a VBScript custom action (type 38) that removes the corresponding rows from the ServiceInstall table.

The poster shows some example code that does not seem to work. Let's look at some approaches to try to solve this problem.

Solution 1

Here's some theoretical JScript to run in a Custom Action that modifies the installer Database:

oDatabase = Session.Database;
oView = oDatabase.OpenView("select * from ServiceInstall");
oView.Execute();
//now our view object has records we can "fetch"
oRecord = oView.Fetch(); //get the first record from the query
while (oRecord != null) //make sure we are not at the end.
{
//TODO: Either handle the logic on what to remove based
// on the oRecord here or in the query. Assumes all
// records matching the query are removed.
oView.Modify(6,oRecord); //6 = msiViewModifyDelete
//get the next record from the query
oRecord = oView.Fetch();
}
oView.Close(); //close the view
oDatabase.Commit(); //commit our changes

Looks pretty good, right? We are opening the database, fetching records, modifying the records, and committing the changes. Sad to say even though the API logic is correct, this won't solve the problem at hand! Why?

Custom actions can only add, modify, or remove temporary rows, columns, or tables from a database. Custom actions cannot modify persistent data in a database, such as data that is a part of the database stored on disk. (View.Modify documentation excerpt)

As an aside, you could substitute the following 2 lines for the first one and modify the msi (permanently) outside a CA:

oInstaller = new ActiveXObject("WindowsInstaller.Installer");
oDatabase = oInstaller.OpenDatabase("myInstall,msi",2);

Solution 2

So, with this restriction, how do we solve your problem? The answer lies (buried) in the View.Modify documentation. The action type msiViewModifyInsertTemporary is our friend. Instead of solving the problem by deleting the services you don't want to install, we can add the services you DO want to install at runtime! Remember to delete the lines from the ServiceInstall table if you use this method

Lets look at an example of this:

oDatabase = Session.Database;
oView = oDatabase.OpenView("select * from ServiceInstall");
oView.Execute();
oRecord = Installer.CreateRecord(13);

oRecord.StringData(1)="MyService";
oRecord.StringData(2)="MyServiceName";
oRecord.StringData(3)="MyServiceDisplayName";
//SERVICE_WIN32_OWN_PROCESS
oRecord.IntegerData(4)=16;
//TODO: continue for the rest of the columns in this table. See
// the ServiceInstall table docs for more information

//msiViewModifyInsertTemporary the record.
oView.Modify(7,oRecord);

oView.Close(); //NOTE: no need to call commit..

You do need to deal with the uninstall issue using another CA. That is not covered here, but the concept is similar.

As an aside, I use the above technique to adjust the ReserveCost table when I need to create a large database. If the DB exists, I do nothing, otherwise, I add the size of the database to the table for proper costing.

Solution 3

Although Solution 2 will solve this problem, why would you want to go through that complexity when there is a much better (and simpler) method built into the MSI engine? In the ServiceInstall table, there is a Foreign Key into the Component table. If the component is to be installed, so is the service.

Simply create a hidden feature, assign the components necessary for the service to that feature, and conditionally install (or not install) that feature.

Why can't I use WScript in my MSI Custom action?

This is a fairly frequent question, and one that deserves a decent answer. VBScript and JScript custom actions run inside of the Windows Installer engine - in technical terms, the MSI engine is the scripting host for all scripts running as a custom action.

The scripting host program can make available objects to the scripts if it so desires. I've written applications that host the scripting engine, so this is familiar to me. In the case of MSI custom actions, the Session object is an example of this. When you run a script from the command line or by double-clicking it, it is hosted by the "Windows Scripting Host" (WSH), and it kindly supplies the WScript object. The main reason these objects are provided is so the script can interact with the host environment. In the case of WSH, you usually need to interact with the user since it is being run from Explorer or the console, hence the "Echo" method. In the MSI world, you are encouraged to interact with the MSI engine and let the MSI engine interact with the user based on the appropriate UILevel settings - this is all handled for you if you use the Session.Message API's to interact with the user or to write log files.

To summarize, if you have a script that contains WScript.something, it will not work in an MSI, as the WScript object does not exist and you cannot create it.

An unrelated issue (but people think it is related...) is the "WScript.Shell" object. You can create the WScript.Shell object (in JScript foo = new ActiveXObject("WScript.Shell")) and use it. In this case, the full name of the object you are creating is merely WScript.Shell by coincidence, and you are not calling a method on the WScript object, as you are in a line similar to WScript.Echo("foo"). An interesting related aside here is an article detailing the differences between CreateObject and WScript.CreateObject.

Danger, Will Robinson! Some antivirus programs and other script blockers will either silently block or loudly proclaim attempts to instantiate the WScript.Shell and/or the Scripting.FileSystemObject. These are commonly used by malware since they can access the registry and filesystem. Write a C++ dll custom action if you need to interact with the system in this way.

A fairly common question posted to the Windows Installer boards is "How do I sleep?" or otherwise delay the script. While I don't have an answer you want to hear, there is an excellent article about the subject here. BTW, the answer you don't want to hear is to write a DLL custom action that merely sleeps, add it to the CustomAction table, and in your CA script call Session.DoAction()

Custom actions that should be standard actions

On the MSI front, Aaron Stebner from Microsoft posted a list of Common custom actions that should be standard actions. It is a fairly good list. Please check the link and add to the list!

Aaron is quite correct that writing a solid (read best practices compliant) custom action is difficult. Number 3 on his list is "Create user accounts." If you need to do this, make sure to check the Platform SDK, as well written sample code for this action is included.

I'm going to head over and add my two cents to the list...

VBScript (and Jscript) MSI Custom Actions (don't have to) suck

One of the greatest things about blogging and its popularity (especially at Microsoft) is that you can get solid information and advice straight from the horse's mouth. In the Windows Installer world, the sole blogger at Microsoft (that I am aware of) is Rob Mensching. A recent post by him entitled VBScript (and Jscript) MSI Custom Actions suck was one that struck some discord with me.

Script custom actions do not have to "suck," contrary to what Rob points out. I prefer to use the KISS concept. There are several concepts that ANY developer, REGARDLESS of using script or C++ Custom Actions must understand.

• Debugability - If this is a real word, it would mean the ability of an individual to debug the Custom Action. In C++, use of DebugBreak() after attaching the debugger to the process is a simple way. In script, I have not found any way of attaching a debugger to a Custom Action (if you know of a way, that would be a great feedback point). In either case, it is difficult, if not impossible, to debug a Custom Action gone bad in the field. Turning on logging, with judicious use of Session.Message is one tactic regardless of Custom Action language. As an aside, I don't understand why the MSI does not allow the author to specify some minimum level of logging.
  
• Understand your Lowest Common Denominator (LCD) - If you are attempting to instantiate an ADO object on an otherwise vanilla NT4 machine, you deserve what you get - a failure. If the application you are trying to install requires something, make sure it is already installed via a LaunchCondition, or that it is going to be installed. If you depend on it in a Custom Action, make sure the install action takes place BEFORE you attempt to use it. In the C++ world, a missing dynamic link library such as MFC could get you into trouble...
  
• Use the right language for the job - Lets say you ignore Rob's advice and make a script custom action to use ADO. Some ADO objects take variables by reference. VBScript supports this, JScript does not. This is no different from the debate on which language you used to code your application. You can code yourself into a corner using the "easier" languages, but on the other hand C++ can sometimes be overkill.
  
• Other Considerations - With the plethora of third party software and networking techniques in use today, a developer must be fluent in new technologies. It is uncommon for a system to have no virus/worm protection. Products such as Symantec's Norton Antivirus have script blocking technologies to block against certain objects being instantiated in script that are typically used in worms and viruses. Instantiating the FileSystemObject will likely fail on systems with this type of protection. If your script is dependent on actions or objects outside the Session or Installer object, I would suggest using only C++ CA's to ensure the installation does not suffer in the future. Additionally, some firewalls prevent applications from accessing the internet (even XP SP2). If your setup uses or relies on these capabilities, be aware of the target audience and its possible setup.
  

Rob's assertion that a robust script Custom Action cannot be written is indefensible. I can (and surely have) written unrobust C++ Custom Actions. The beauty of software development is you can be as robust as you want to be, no matter the language. You, as the Custom Action author, are responsible for providing a meaningful error messages that can direct the user to the appropriate action to get the installation to work correctly.

To play Devil's Advocate, one point against script custom actions he didn't make include obscurity. Since MSI is easily decompileable, and the scripts can easily be viewed, edited, and replaced, it is likely there are some things you would like your competition to devote more time to reverse-engineering.

InstallShield provides its own scripting engine that can be used in Custom Actions. Although there is additional debugability and no antivirus blocking concerns, be aware that the size of the MSI increases (it has to install the scripting engine), and you are locked into InstallShield until the action is rewritten. I was able to quickly migrate almost everything from Wise for Windows Installer to InstallShield because I stuck to the MSI standards and ignored Wise's custom functionality. Better yet, I can move back if Wise leaps in front of InstallShield.