Transacting business over unverified email addresses = Not Smart

On April 21, I recieved an email from 'SGW Payroll' [sgwpayroll.com and paydashboard.com]:

Welcome to the SGW Payroll Portal [Redacted First Name]
A new employer admin account has been created for you, either by one of your colleagues or by us at SGW Payroll Ltd.
Please click on the following link to choose a password and sign in to your account. [Redacted Link]
You are receiving this email as your employer uses PayDashboard to deliver your payslip. Click here to manage your email preferences.

This welcome email looked phishy, so I checked the headers, then the payroll site, and it looked legitimate. I'm in the US, this company is in the UK. It is pretty obvious this was a mistake - likely bad entry of email address, but note the lack of any method for me to indicate there was an error. Therefore I ignored the email hoping that the intended recipient would realize they did not get the expected welcome email and correct the problem.

On April 25, I get another email:
SGW Payroll Ltd has published new documents for [Redacted - Name of Business] within your document storage hub. Log in to view or download the documents.

This was bad.  Looks like there is now an active account - mind you, an 'employer admin account', that somehow, without clicking the link, they started to use.  I also now have the name of the business.  At this point, I decide this could be a real problem, so I forward the email to support@paydashboard.com with the message "I did not request this, it is likely someone mistyping their email address."  I get an automated response, then a reply from "Paul Gibbons" from their support: "Thank you for your email. We have started an invesitigation [sic] to have this removed. Thank you for bringing this to our attention." Awesome. Case closed.

On May 21, almost one month later, I get yet another email - same template as the prior one.  I respond back to the support request as follows:

I received another email today. Please resolve this. I am not the intended recipient.  There is no 'investigation' required. You are in the UK. I am in the USA. I do not work for this [Busness type redacted].  At this point I consider this to be:

1. Horrendous security on your part.  Any email address entered into your system should have a 'confirm' step where you send an email to test the ability of the recipient to receive an email. They click on a link and log into your system. Only then should you consider the email address 'worthy' of use for anything sensitive.
2. Even MORE Horrendous security on your part. Someone sending you a reply saying they are NOT the intended recipient should IMMEDIATELY result in removal of the address, flagging it as suspect.

Please have a supervisor/manager contact me immediately. I really do not wish to engage in a public humiliation campaign against your company. You are sending someone else payroll information!!!!!!!!!!!!!!!  This is crazy!!!!!!!!!!

I get a response, again from Paul Gibbons: "You [sic] ticket has been escalated to myself to review. I will speak with the agent populating our software to establish what is happening and will revert back."  That's awesome. I ask for elevation, and the SAME GUY elevates the issue to HIMSELF.

I decide to take a closer look at the company's website to find contact information for someone outside of support. No luck.  I now have a name of their CISO but am unable to find an email address at the corporate level or via linked in.  I reply to Paul: "Paul, you 'reviewed' it the first time, and did not correct the issue. Please have Jeremy Lloyd contact me, ASAP." To which I received the reply from Paul: 'We will respond once the investigation has been completed.' I give Paul one more chance to fix this.

On June 1, I recieved another email:

There are a couple of things you might want to take a look at
Hi [Redacted - First Name]

As we are able to compare data based on your employees payslips, we want to let you know when we spot something we think you should take a quick look at.

When you log in to your Employer Dashboard you will see "Notifications" at the top right of your dashboard. Click on the link to view the latest notifications for your company. [Redacted - Name of Business, links, rest of email, etc]

In response, I will send Mr. Gibbons a link to this post.

Since this saga began, I have received several other emails obviously intended for a person in the UK with a similar email address to mine.  From the information in these emails, I can tell you full names, address, purchases, restaurant reservations, bed and breakfasts where they spent their holiday, etc. I can also tell you that no entry point into various systems required even a simple 'click to confirm this is you' email address confirmation. I could have cancelled a dinner reservation with a simple click.  It sounds a bit scary, but I have to believe most people would do the right thing. 

Advice to developers and analysts, especially with GDPR:

1. At a minimum, you need a business process to handle a 'I got this email by mistake, it is not intended for me' response from a user. Always. It could be completely manual. But this DOES HAPPEN.
2. At any entry point into your system - a welcome email or similar - provide a link to initiate that 'incorrect email' business process. Every email from that first contact until the email address is determined to be valid should have a similar link, and NOT disclose personal information. Only consider an address to account valid when a link clicked from an email (or a code sent only via email) is authenticated as being from that user.  This does not need to be a login per-se, but a text message from a registered phone, etc. could all be valid. 
3. An unconfirmed email address should never be the sole means by which you reset a password or create an account. [NOTE: I did not try to see if I could do this with PayDashboard, as I consider that to be fraud. I would be surprised if I couldn't reset the password].
4. Some email providers ignore punctuation, like gmail.  Others also allow automatic aliasing, such as gmail, where you can add a "+" with other text and the plus and text are ignored.  This is great for users, as I can filter emails based on this. However, accidental creation or intentional creation of multiple accounts is possible based on wont of a period.  This is something to be aware of, I'm not sure how actionable it is.
5. You should publish an email address or a method of contact for the executive team of your company. It should be monitored and legit messages forwarded. Sometimes through poor training, incompetence, or merely one person having a bad day, an interaction with your company WILL go poorly. You should give a method to reach the executive team so that these can get handled sooner rather than later.

SOPA, PROTECT-IP, and Legislative Position Statements

At the end of December 2011, I wrote to my federal representatives regarding my opposition to SOPA (US House) and PROTECT-IP (US Senate) legislation that portends to reduce piracy and theft of intellectual property.

Today, many websites are protesting these bills. Wikipedia and others have decided to shut down for the day (or part of the day).  Google and others have decided to alter their homepages.

I encourage you to view the legislation via the links above for a Wikipedia summary, then follow the 'External Links' to the text of the bill and develop your own opinions.  If you wish to oppose it, Google has a petition online that you can sign.  In either case, you should also write your representatives directly.  Hopefully, you will get a useful response.

Senator Toomey (R-PA) only responded to me with a confirmation that the legislation does, in fact, exist.  I did make specific references to provisions in the bill, so it is obvious that I already knew this. A follow-up did not result in an actual reply regarding the Senator's thoughts on this bill.  Senator Casey (D-PA) and Representative Altmire (D-PA) both failed to respond to my email.  I only give minor props to Toomey for responding at all, however, all three of my representatives get failing grades for their lack of adequate response.  

Even if my representative has opposing viewpoints or is projecting a vote contrary to my opinion, I believe it is their responsibility to have and disseminate intelligent position statements on each piece of legislation pending in their chamber and inside any committee that they are a member of.  An intelligent position statement is one that is published within 48 hours of the bill successfully leaving the committee and would include all of the following elements:

• A link to the full text of the bill, including chronological history of successful amendments with time stamps.
• If the vote were held today, based on version at timestamp ???, I would vote (yea/nay/uncommitted)
• In the words of the representative, a summary of the intention of the bill.
• List of elements that the representative supports and believes critical to the success of the bill.
• List of elements that the representative opposes.
• List of elements that require additional flushing out or personal research.  This list would be required if 'uncommitted'.
• List of bills pending in committee or in the queue for a floor vote that portend to address the same concerns.
• List of existing legislation on this topic and established case law that covers (or fails to cover) the subject matter of the new bill.
• Media reports, studies, corporate statements, lobbyist groups, etc. that advocate for the need of the new law.

Update 1/18/2012: Sen Toomey released a statement that he does not support SOPA or PROTECTIP "in their current forms", yet he fails to make any statement as to what specific portions of it he takes issue with.  This statement is clouded in doublespeak.  Please keep the pressure up to get him to explain his position.

Wix, Votive, and Semicolons...

If you are using the Wix in Visual Studio (known as 'Votive') and need to set one or more preprocessor variables, it is rather simple. If you right-click on the Wix project, select 'Properties', then the 'Build' tab, you simply populate the 'Define preprocessor variables:' text box like so:

Name1=Value1;Name2=Value2

If you are using MSBuild, or editing the .wixproj file itself, this translates to the contents of the 'DefineConstants' element, which is where Votive stores what you put in that text box.

Things, however, are not really clear (or documented) if you need to set Name1 in the example above equal to a semicolon delimited list - for example "one;two;three" - so lets try it this way:

Name1=one;two;three;Name2=Value2

Candle.exe is passed (which is obviously incorrect based on our intentions):

-dName1=one -dtwo -dthree -dName2=Value2

The solution is NOT to put quotes around the list (my first guess), but to replace the semicolons that break up the list (and only the ones in the list) with '%3b', like so:

Name1=one%3btwo%3bthree;Name2=Value2

Candle.exe is now correctly passed:

-dName1=one;two;three -dName2=Value2

I do not know if this is way you would handle this situation in anything newer than Wix 3.0 - I haven't updated to 3.5 yet.

Electric Power Generation Choice in Pennsylvania

This post is centered around the deregulation of electricity generation in Pennsylvania, why you get to shop for an electricity generator (the history lesson), the factors to consider (the practical lesson), and ultimately how to save real money (the lesson in pragmatism).  I'm sure that the general advice applies to other states that have started 'deregulation' of the electricity market.

Let me provide some quick background on the process by which electricity gets to your home.  There is, of course, a power plant that makes electricity - this is called a 'generator'. There are several 'generators' that are connected with each other at multiple points - this is known as the power 'grid'.  The purpose of the grid is to assure service in the event that one or more of these 'generators' is turned off or disconnected that power can still be supplied from other generators.  It made sense for generators on many levels to interconnect with other generators owned by different companies into the same grid.  Meters indicating how much power each generator supplied into the grid allowed for proper bookkeeping.  Then you have the 'transmission and distribution' part - which is the lines connecting the generators to the grid, and the grid to your home and meter how much power is consumed.  This is a very simple model of something much more complex.

The way electric power was implemented, one company was responsible for dealing with all parts of the process - even if the company that billed you had no actual generation capabilities.  You were charged one rate based on how much power you consumed.  These companies are a 'natural monopoly' - mostly because it doesn't make sense, economically, to run multiple power lines from several companies in parallel. Because of this competition does not exist.  This is a similar situation to gas and water companies today.  Over a decade ago cable and telephone companies were in the same boat, but technology advanced to the point that both wiring systems can carry signals that allow for a limited amount of competition.

In 1997, a PA state law "Electricity Generation Choice and Competition Act" was passed (Regulations covering this law at 52 Pa. Code § 54 - pdf).  The links provided are to the text of the current law and rules and regulations pertaining to it, as amended.  The intent of the law was to deregulate the electricity generation market.  Since any generator in close physical proximity to a consumer can apply power to the grid, and the amount each one applies can be controlled, why should the consumer be stuck with buying power from one specific generator?  The direct impact of the law to the consumer consists of a few major points: (1) The costs of generating power were separated from the costs of delivering the power from the grid and itemized on your bill.  (2) All of these rates were temporarily capped and tightly controlled, until (3) These caps expired on Dec. 31, 2010. The very first part of the law, Declaration of policy, describing its complete intent, is a good read that is easy to understand.

The law attempted to take into account consumer protections and industry protections during the transition period to an unregulated market. If we look at the old regulated market, the supplier either engaged in long term contracts with a generator or generated the power themselves.  Unless you (the consumer) were willing to build your own power grid, you were at the mercy of their business decisions. These decisions were based on the monopolistic system of the time, industrial/residential growth projections, understated nuclear power costs and growth, and should be viewed in that light. Some were good, some not so good.  The not-so-good decisions resulted in what was called 'stranded costs' in that if the market was opened up and the utilities were forced to sell power for their cost of generation no one would buy power at that price.  After the law took effect, the generators were able to recoup these 'stranded costs' but under a capped price system - essentially meant to prepare the generators to compete in a free market. A whole book could be written explaining the theory of how this works. In the end, it sort of worked out - the 2008 national cost of electricity was 9.83 cents per kWh and PA was 9.60 so the 'capped' numbers were not far off in the end.

What the law didn't take into consideration is that deregulation does not and can not do anything for supply capacity and demand in this particular market. There has been very little supply capacity added to the market since the death of domestic nuclear power.  This law did nothing to make it easier to add generating capacity. Government projections show little growth in capacity. Power generated by your former monopolistic provider can now be sold to other distributors both in and out of state at competitive rates further reducing local supply by filling demand elsewhere.

Electricity, like oil, is a 'source unknown' commodity.  An electron is an electron regardless if it came from solar, wind, oil, coal, gas, or some guy on a treadmill.  The true difficulty in the marketplace now is figuring out who owes whom what... The whole concept of the electric grid is that it gets fed (hopefully) at the same rate it is drained - figuring out who owes who for the times you over or underfed the grid sounds like an added task  (and challenge) for your local utility.

By making your selection of generator, you are telling your utility to buy the same number of kWh that you use from the generator you selected at the price you are contracted for.  There are several selection criteria that you may want to consider aside from the current price: (1) Environmental reasons such as how that generator produces power, (2) Rate terms - how long you wish to 'lock in' a specific rate, (3) Usage patterns - can you or are you willing to juggle your energy use to minimize peak load or consumption based on time of day.  Make sure the 'price to compare' you are quoted from various suppliers includes all fees and taxes. The Gross Receipts tax is complex to calculate yourself if not included, and a really stupid tax for more than just that reason.

Environmental concerns is a tough one to compare.  If this is a concern for you I'd suggest rating each generator on a four star system based on their byproducts of generation (coal, gas, oil, nuclear, or renewable) and your personal belief system. I'd personally favor nuclear and renewable equally - but to each his or her own. This information is difficult to obtain from most providers.  Be aware that at times your generator may need to buy extra power on the spot market - exactly how much and from whom and how are all something to concern yourself with.  They could 'trade' power (best) by taking some extra now and paying it back when production peaks - wind and solar is very inconsistent.  They could buy on the spot market for demand peaks (medium). They could be oversubscribed (bad) where they are never producing as much as their customers are buying so a portion of the 'clean' energy you are purchasing is really quite dirty but you are paying much more for it - if those profits are 100% dedicated to expanding their production capabilities and are NOT considered profit then that may be OK.  Environmentally sensitive folks are likely better off conserving and selecting the cheapest provider while investing the difference in for-profit companies researching commercially viable clean energy production.  Funneling money into the current non-viable technologies only slows the progress in developing truly groundbreaking and cost-effective solutions that will truly benefit us all. If you really do the math on what goes into manufacturing (inputs and byproducts) and transporting solar, wind, and battery systems required to support them and amortize that environmental cost over its lifespan they are not as attractive as many believe them to be compared with other options - especially modern nuclear technologies with near-zero waste.

Some suppliers have a contract term where the rate is held constant for a period of 1, 2, or 3 years. These providers may or may not have incentives for signing (gift card, airline miles, etc.) and termination fees if you quit early.  Be sure to add in the costs and benefits from those deals as well in your calculations.  Should you do a 3 year lock in?  The government is predicting generation prices drop from current levels and bottom out in 2012 then begins to rise again, yet historical price data shows steady climb in the 'real' column so you can bet either way here.

There are other ways that you can change your usage behaviors to save more money. Can you go to load based pricing to save money (running dryer, dishwasher, A/C, and stove/oven at separate times to minimize the simultaneous current draw)?  Is there a time based rate plan where you concentrate your power use to off-peak times and pay less for it?

All that stuff mentioned above is rather difficult for the average person to understand and digest.  What happens if you DON'T choose a specific generator? If you don't you will be the sucker that will end up subsidizing the prices that allows your default generator to sell power cheaper to other utilities.  The 'default' generator is assigned based on contract with your utility - when your contract expires with your current generator, they close or get shut down, or you go into financial default you get that one.  There is no incentive for them to charge the lowest rates, since a good number of people won't choose and they are allowed to recoup fees and costs associated with being a default generator.  Based on a brief survey of default vs. cheapest alternative the 'idiot tax' on people that don't pick a provider is 5-10%.  Oh, and the default generator is not allowed to provide a usage based discount but other generators can so not picking can cost you even more than that.

There is a case where you may not want to switch... at least today... so just skip the next three paragraphs. If you have an all-electric home and are under 'Residential Heating' billing codes (stated on your bill as 'RH' - 'Rate RH' and 'Penn Power RH') you are probably charged different rates at different times in the year. This rate system was developed years ago to encourage all-electric homes.  Electric only homes are interesting, because their usage is much greater in the winter than summer - the exact opposite demand cycle of a gas heated home. Electricity generators need to have the capacity to handle peak loads or you have brownouts/blackouts and since the all-electric home is in the minority that occurs in the summer.  In this peak usage time, smaller power plants get taken online or offline based on demand (which correlates with temperature) - these are generally the most inefficient and costliest ones to run like coal, oil, and gas fired plants.  When they are turned off or down you save a bundle as you are burning less fuel. When you have a nuclear plant, it does not saves much money when production is less than the maximum capacity.  With the advent of cheap excess power in the winter, you now want to encourage usage spikes in the winter to get the best rate of return on your nuclear power plant investment.  Enter the all-electric home, which is only cost effective comparable to gas (in our climate) if the cost of electric power is cheaper in the winter.  Deals made with developers to offer a special rate plan where this dream can be fulfilled.  This was a mutual win for homeowners and power generation plants.  People built their homes, chose their appliances, and chose their heating system based on these rate promises.  The history lesson is now over - lets look at what this means.

As of Feb 28, 2011, Penn Power's Residential Heating rates for June-Sept were 6.44 cents/kWh and for Oct-May were 4.50 cents/kWh straight up.  This means no kWh minimums before the discount was applied - just a seasonally adjusted generation rate. According to a phone call I had with Penn Power on 3/4/2011 there is no plans to change their existing program, although no new subscribers can be added to it.  Also of this date there is no competitor for this pricing plan.  If you fall into this boat, as I do, you shouldn't do anything.

PECO Energy has a slightly different program which is being phased out (more discussion on that topic here).  I'd call your provider to get the details and figure out your own cost structure.  Through 3/31/2011 their prices were 9.74 cents/kWh for the first 600 kWh then 5.35 cents/kWh for any usage above that with no mention of summer prices.  Pennsylvania's consumer advocate, Irwin A. "Sonny" Popowski, states: "The commission regulations essentially require the elimination of the special winter heating rates, though we have tried to do this over a multi-year period." I fail to find supporting evidence of this statement in the Rules and Regulations. Since the context of the quote is PECO specific, and their RH rates are discounted only after a kWh minimum is reached, the reference may be to that specific implementation of the winter heating rates by a default provider.  Personally, I call bullshit on this rate change being from the legislation or Rules and Regulations since PECO's price to compare for non RH codes varies in June 2011 (9.99 cents/kWh for the first 500 and 11.20 thereafter). I'd call your legislator and 'Sonny' and get the specific portions of the law/regs I linked to above that he is using to support his assertion. It's PECO's game, anyhow, and yeah - you folks in Philly are probably screwed regardless of what the law says.

After looking at the options across the Commonwealth, I conclude that you could probably save more money by conserving than switching.  Remember that if you save 10% by switching that is only saving the generation charge - not 10% of your total bill.  To learn how to conserve, you need to know your baseline usages, and understand where the power is being consumed then reduce it. This site explains what a kWh is and how to save energy. There are also some simple techniques to check basic insulation effectiveness - like seeing if the snow melts off your roof faster than all your neighbors (that would be bad). Other such tips can be found in the two books I recommend at the end of this post.  If you have electric heat the book on insulation is excellent and a must read.

Good luck in selecting a provider - you will need it.  I found that trying to figure out all the nuances of selecting a provider is way too specific to your particular needs and usage patterns to offer any sort of general advice.

More Information:
PA Office of Consumer Advocate Shopping Guide
PA Public Utility Commission 'PAPowerSwitch' Site

Recommended conservation books:

Wix way should you go?

Now that I know Rob still reads my mostly stagnant blog, I guess it is the appropriate time to write a long-overdue post.

When I started working with Windows Installer technology, back when it was first introduced with Office 2000, I played around with customizing the package for the IT department to push out a customized version. The tools were quite primitive, the technology was new and largely unknown, and the concept of having blogs, yet alone Microsoft folks blogging, seemed completely foreign. Support was pretty much nonexistent, and much of the documentation was unintelligible. Fast-forward 10 years, and what a difference that makes! Today there are several free and low-cost repackaging tools for transitioning non-Windows Installer based setups to the MSI format, authoring tools, and lots of community support.

Most setup authoring tools have significant issues. Non-MSI or script based installations have issues because they encourage hacks - I can't tell you how many installations I encountered that install services by writing keys to the CurrentControlSet hive and forcing you to reboot merely so the Service Control Manager can pick up that addition. Furthermore, if you are targeting any sort of enterprise where more than one of your setups will be installed IT departments want MSI deployments for very good reasons. GUI based Windows Installer tools fail to do a good job of grouping related things into the same component, and dynamically adding a directory of files at build time breaks patching semantics horribly. Another big disadvantage to these tools lie in the setup author because he or she does not need to understand the underlying technology and can get away with "programming by coincidence" (as described in The Pragmatic Programmer).

I remember several paradigm shifts throughout my experiences with setup technology - nested MSIs, merge module distribution, and chaining installations. During this time the stock price of Rolaids likely skyrocketed. The biggest challenge was attempting to get developers to take a more proactive approach to deployment considerations as they were writing their code. One approach that I took was the use of merge modules - developers of feature-units would package their build output in an MSM that was consumed when building the final product. Using Visual Studio 2005+ with their deployment projects was not only difficult, but downright impossible because of how limited, shortsighted, and buggy deployment projects are. Adding custom actions to these modules involved a complex and convoluted post-build scripting process that nobody understood, but it DID move teams towards the direction of thinking of deployment while coding.

These days, the tag-team of MSBuild plus Wix 3.0 is THE enabler to accomplishing those goals and largely eliminating the disadvantages of the GUI-based tools. Since there is close to a one-to-one correlation of XML elements to the Windows Installer tables, it is quite simple to follow if you understand the underlying Windows Installer engine. To use WiX to author a complete installation, you MUST have an understanding of the Windows Installer engine. To make a few tweaks or additions once the basic skeleton of the installer is laid out, just about any developer can do it provided access to the WiX documentation. I have team members that are NOT setup developers add services, event log sources, and more with no official training.

Some of the more compelling points in favor of WiX is how you can use it to easily and properly make multiple product editions which share components, separate units of related components into their own WXS file(s) for easier understanding and maintenance, and integrate it easily as a first-class citizen into an MSBuild project. No other product is available to my knowledge that accomplishes those goals. Best of all - WiX is free, fast, and easily installable onto any developer machine.

If you are looking to switch authoring tools, take WiX for a test run by using the dark.exe decompiler to convert your existing MSIs and play around with it a bit. Subscribe to the WiX mailing list and ask a few questions. You just might like it.

Congratulations to Rob and the entire team and individuals who have contributed to it, as well as the community of developers who support it via the mailing list on a daily basis. If you are ever in Pittsburgh, let me know. I'll buy you a beer.

Visual Studio 2008 GenerateBootstrapper task and UAC

I am extremely disappointed in the Visual Studio 2008 bootstrapper to say the least. While the concept of a bootstrapper is great, it is obvious that the designers did not look at real-world deployment scenarios.

Let me give one example scenario: Installing prerequisite packages and interacting with Vista's User Access Control (UAC). Most of the time, the author of the bootstrapper (by the way that packages are authored and selected) is pretty confident as to how things are required (and I use the word required here very specifically) to be installed: per-user or per-machine. In some cases, such as the .NET framework and many third party redistributables, this is a per-machine only situation. Perhaps some components could be installed per-user, but generally if one part is installed per-user, then all parts capable of being installed per-user should be.

The above understanding should certainly be incorporated as part of a bootstrapper from the start. If some components require a per-machine installation, UAC should be prompted for once, start the elevated process, and this elevated process could be 'commanded' by the non-elevated bootstrapper main process. If per-user components exist, the option to install all of them could be given to the user, and appropriate ALLUSERS=x or command line arguments could be passed to the chained installations capable of doing per-user installations, and the elevated or non-elevated boostrapper process is used based on the requirements of the package and user selection.

Instead, the VS 2008 bootstrapper prompts for elevation of each chained installation, even though the entire POINT of a bootstrapper is a single entry point and interaction for the user to get a piece of software installed onto a users machine. We as developers consider setups a massively coordinated symphony of prerequisites and third party components. The user considers software they purchase or wish to install as a single piece and a single process, and it should seem this way to them.

For setups that are rather long and/or chained-installation heavy, prompting after each chained installation is aggravating to the user and is completely unnecessary from their perspective. This design flaw requires the user to babysit the entire process. We as developers hope that the user chooses to elevate each one - if not the entire setup fails with a rather hopeless error message requiring a support call.

Corporate users are the only ones who generally care about per-user installations, and generally can follow a dependency chain of installations to push apps to the users. Most of them will NOT use our generated bootstrapper, anyway.

The point of this post is to tell you how to work around the 'elevate for each' behavior of the generated setup.exe by hacking the setup bootstrapper used by the MSBuild GenerateBootstrapper task so that it prompts for elevation once at startup and installs everything elevated. This is not ideal because sometimes it is not necessary to elevate, but it certainly makes the end-user result better for the 'happy path' for 99% of all users.

Normal caveats apply - if you break something, its not my fault. Keep a backup of all modified files. The solution presented here will change the behavior for ALL GenerateBootstrapper build tasks unless as part of your msbuild script you replace the appropriate file with the version that gives you the appropriate results.

The Windows SDK actually contains this bootstrapper, and it can be located here on a default installation on a 'normal' PC: C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Engine\setup.bin. Copy this somewhere and rename it to setup.exe. In Visual Studio 2008, do a File->Open->File and find your setup.exe. This will open up a resource viewer/editor. The RT_MANIFEST 'folder' is what you want to expand. Right-click on "1" and export it, saving it as a text file. Open the text file and find the requestedExecutionLevel node - change the level attribute's value to 'requireAdministrator'. Use CTRL-A and CTRL-C to copy the contents of this file to the clipboard. Back at the resource editor, double-click the '1' entry to get it in the hex editor. Select all but the first three bytes and paste your clipboard contents in there. Save and exit. Now rename the setup.exe back to setup.bin and replace the original file, saving a backup, of course. The next time your build happens, the setup.exe will now use the new manifest, giving you one prompt and doing EVERYTHING elevated.

I won't get into the other issues with this bootstrapper - no autogeneration of MSI logs, no timestamps in its own log, immediate run and exit that hinders seld-extracting archive packages for single-file download, no MSI caching for patch scenarios like the Office team uses, spanning over multiple CDs/DVDs etc. I will wait for Wix 3.5's Burn to hopefully resolve these issues and the ones not mentioned. Rob Mensching, are you listening? I'll even volunteer to help write it.

Turning off Data Execution Prevention (DEP) for IE7 on Vista x64

Time flies so fast that you dont even realize you have not written a new blog entry in a very long time...  

In January of 2007, I posted a comment to a security blog relating to the difficulty in turning off DEP in the 32-bit version of Internet Explorer 7 on a Vista x64 OS.  As to why you would want to do this, I will offer no opinion and would not recommend it in general for long term use.  However, lets say that your 32 bit IE (on Vista x64) opens and immediately crashes for some reason.  This means you need to tweak settings (like removing an add-in) for the 32-bit IE, yet the control panel only offers the 64-bit IE settings. Normally, you can get to the 32-bit settings from inside a 32-bit browser instance but due to the immediate crash this is not possible.

A very astute individual came across my comment and since comments were closed on the initial post, sent me a how-to guide for accessing the 32-bit IE settings in Vista x64.  I offered to re-post the details here with attribution.  Kudos to Razvan Socol, a SQL Server MVP, for providing this solution after encountering the same problem (after "blindingly following the advice in the Security Advisory 961051"), which makes this post fairly timely in case someone else falls into this trap:

To access the options for the 32-bit Internet Explorer, I started an elevated command prompt and executed C:\WINDOWS\SYSWOW64\EXPLORER.EXE /SEPARATE to execute Windows Explorer in 32-bit mode. In this window, I navigated to the Internet Explorer icon on the desktop and by right-clicking it, I accessed the Internet Options for the Internet Explorer 32-bit, where I could finally uncheck that box.

 

I'd like to post a comment on Michael's blog to share this info with the world, but the comments are closed for that post. Anyway, I thought at least I should let you know on how to solve that problem, although I'm sure you have solved it in another way by now... [ed: I actually did not find an easy way to do it previously outside of several registry tweaks]

Software Product Lines

This is the first of a series of articles on an approach to systems architecture called “Software Product Lines.” To introduce the concept, let’s ignore the word “software” for now and focus on what a product line is in a more general and tangible fashion.

If you are in the market for a laptop and go to Dell’s website, you have a choice of several different base models that can be configured to your specification. Once you select the base model, you can configure things such as the size of RAM, size of the hard drive, type of display, etc. What makes this possible is a hardware architecture of various components that can fit into the same slot – compatible electrically as well as physically. This allows Dell to create and mass produce (or purchase) several standardized components and simply plug them together to create a customized product at a reasonable cost in a short timeframe. When new technologies come out that require changes to the base product, like ATA laptop hard drives, Dell spins up a new product line with that one modification reusing the old technology and components that is still valid, tried, and true. This sounds like a great idea and solid common sense, doesn’t it?

As with anything, there are challenges to this approach. Take for instance a design flaw in one of the more heavily shared components, such as the motherboard, where the power connector solder joints go bad due to heat and constant plugging/unplugging. This one design flaw could end up costing dearly in repair and possibly recall expenses, as a significantly higher number of devices are impacted then if the system was not as modular. It is also significantly more difficult and time consuming up-front to design, build, and test a system that shares components. In a product line, tight adherence to specifications needs to be considered for future compatibility. Take for example the wireless network/Bluetooth riser cards found in many laptops. Let’s assume the bus timings are just a hair off spec, but the current crop of hardware is tolerant of it. Newer technologies coming out six months later may not be tolerant. A non-product line approach would be to make a revision of the entire system – which is required anyway; however, the product line approach requires significant expense to revise and test one or both of the components to assure compatibility.

Software is not very different than hardware when it comes to product lines. Most high-end software packages are heavily customizable so it can integrate into a variety of environments. Microsoft’s SharePoint product is one example, SAP is another. These two products are off-the-shelf packages intended to appeal to a very large market. There was heavy up-front investment by the vendor in design, documentation, interfaces, components, external training and certification programs, and more. But what if your product is very specialized with a small market? Even worse, what if your product is designed for small to medium sized businesses that typically can’t afford or will not have the resources to self-customize such a vast system? To top it all off, certainly, you cannot afford to invest in the amount of up-front work required to build, test, and debug a heavily customizable system given the small size of your target market.

Enter the concept of Software Product Lines. It is important to note that this is an engineering approach to solve a specific business problem. Using the Software Product Line methodology in a business environment that is not controlled or the problem domain of the customer is not well understood will cause you to paint yourself into a corner with no easy or cheap way out. Another salient comment is choosing this approach should not significantly change your development methodology (assuming, of course, that you have adopted a legitimate methodology in the first place). Finally, this approach is NOT a method of implementation, merely a way of thinking from a design perspective of what needs componentized. The decision to use a Service Oriented Architecture to implement a Software Product Line is off-topic – these are two different animals entirely.

Part II of this series will delve into some of these concepts deeper.

Go Steelers Redux

Not only has it been a long time since my last post, but it has been exactly a year since my last post on my hometown NFL football team, the Pittsburgh Steelers. Sadly, last year we lost the AFC championship. This year we won it against the Denver Bronco's. This means we are off to the Super Bowl against the Seattle Seahawks. Superbowl XL is on February 5th, 2006. In preparation, my blog banner has been updated to "All Steelers, all the time" from the revolving banner of the past.

One of the great things about living in (or coming from) the Pittsburgh area is the friendliness of the people in and around the city. Last year, I blogged a bit about Myron Cope and the "Terrible Towel." One thing I neglected to mention is that all proceeds on the sale of the towel go to the Allegheny Valley School that provides care for children, adults, and seniors with mental retardation and physical disabilities. If you would like to support the Steelers, and the school, please use the above link (instead of the Google AdSense ones) so that the school gets additional credit.

I believe the Steelers have been playing for the 'Bus' - a.k.a. running back Jerome Bettis. The Bus has been a Pittsburgh fixture for years, not only as a dependable player, but as an excellent example of what a celebrity should be like. To illustrate my point, Hines Ward (a teammate and Steelers Wide Receiver) was in tears last year after the AFC Championship loss because he believed the team let Jerome down in his potentially last season. This year is different. Jerome will have the opportunity to play, and in his home city, Detroit. Jerome is active in the community, and is currently raising awareness about Asthma, a disease he is afflicted with, in addition to a program called "The Bus Stops Here," benefiting children from both Pittsburgh and Detroit. Jerome's parents, Gladys and John, have every reason to proud of their son, and the values they instilled in him.

Other team members have had tremendous impact on the Pittsburgh region. I don't want to leave out any players, but Charlie Batch, a backup quarterback, is also famous for not only coming from the Pittsburgh area, but for his contributions to it. There are several other examples of how the team and its players gives back to the community.

I believe much of the reason for the positive impact of the players of the Steelers goes back to the team's ownership and philosophy. The most famous owner is Art Rooney, a.k.a. "The Chief", who bought the franchise in 1933. If any Pittsburgher's have not yet seen the play "The Chief" at the Pittsburgh Public Theatre, you don't know what you missed! This play ran in 2003, 2004, and earlier this month - keep checking for it to return (possibly) next fall at the PPT website. Franco Harris (star player in the 70's) was talking about Art Rooney when he said, "He was always there to help and to give. And this feeling filtered down to the players. I think the Steelers' players give more to their community than any other team in professional sport."

I'm hoping that the Super Bowl in a few weeks does not bring on any additional heart attacks among the fans, but that you watch it with the understanding that many of the athletes on both sides of the ball are using their celebrity status and monies to better their communities. I think we can all learn from this.

To my readers in the Seattle/Redmond area - I didn't mean to leave out the players of your team, I merely am not familiar with their good works. I am, however, familiar with Victor's Celtic Coffee Company, and am currently out of the "Redmond Slough" blend as of this morning. I'd be willing to trade for some of that in exchange for posting Seattle Seahawks charity links and an equivalent amount of coffee from one of our local coffee houses (Coffee Tree Roasters)...

Go Steelers!

Silence is a virtue...

I was looking for some way of explaining my recent blog silence. As one who likes to use the words of others to explain things that are difficult to explain in my own words, I hit the quote book and encountered this gem by Sally Berger: "You never saw a fish on the wall with its mouth shut." That doesn't really fit, but I found it kinda funny and that will have to do.

In addition to the typical excuse of the holidays, I also recently changed employers. This has the side effect of changing the type of development I do. Keeping with my philosophy that I speak only for myself and not my employer, I will not divulge the name of my current employer. I will also not divulge anything that could be considered something other than generally available developer knowledge. So in reality, the only thing that could possibly change is the type of content.

Most likely I will be delving into topics such as .NET Development, the Compact Framework, and native Windows Mobile (Windows CE) development/debugging. Some of these postings may be about deployment concerns in the Windows Mobile environment. Content suggestions related to this area of development are welcome. I also plan on finishing up a few posts on MSI related subjects I started a while back after some editing and reviewing.

If you have questions regarding MSI of other installer technologies, please keep them coming - many of the privately asked (and answered) questions I have saved as blog post drafts to be completed later, and I will continue to attempt to answer them time permitting.

In the meantime, constant reader, please bear with me while I adapt to my new environment and gather together some new (and old) content.

Writing Clear Code

This topic is a bit of a divergence from Custom Actions, but useful for some general coding skills. I'm going to highlight the C# language in this post, although you don't need to know C# to benefit from this discussion - I'll cover the essentials in the post. The topic is how to write clear code that makes it easier to defer the understand the intention of the code rather than the behavior of the code. The subject came up about a few months ago in a discussion with a coworker who initially didn't quite agree with me. I'll lay out the argument here as I did with him.

Take for instance the following code snippet:

string a = "hello";
string b = "hello";
System.Console.WriteLine(a == b);
System.Console.WriteLine((object)a == (object)b);

Lets explore the last two lines. The a == b equivalence operator is working on strings, and by C# rule, two strings are equivalent if they are both null, or if both values are non-null references to string instances that have identical lengths and identical characters in each character position. Translation - two nulls are identical, and any two strings that match, case sensitive, will result in a true expression. In the latter comparison, when we cast both of the strings to objects, we are not comparing the values of the strings, but the objects themselves.

What do you think it will print? The answer is True and True. Why? String literals that are identical within the same assembly refer to the same instance. Translated, 'a' and 'b' are variables that refer to the same underlying object.

Now lets replace the declaration and assignment for string b with:

string b = "he"+"llo";

Now what do you think it will print? The answer is again True and True. Why? Because the Lexical analysis (lexer) in the compiler stripped out the needless additive operator in the literal. The same would have happened if we changed the string to be "hell\u006f" - since '\u006f' is the Unicode escape sequence of the lower-case letter 'o', and this expansion takes place (most likely) prior to the lexer during the transformation phase in the compilation process.

Now lets replace the same line as above with the following two lines:

string b = "he";
b += "llo";

Now what do you think it will print? The answer is now True and False. Why? The strings are equivalent, but since the preprocessing of the source file before the compilation did not identify the strings as being the same literal, they are in different objects. In fact, because a string is immutable (once created cannot be changed), there was the construction of the string "b" during declaration and initial assignment that was later garbage collected when a new instance of "b" was created during the string concatenation.

Now that the necessary background information is understood, can you tell me the intention of the following code snippet?

string a,b;
//code here that assigns and manipulates a and b
...
if (a==b)
    compareOK();
else
    compareBad();

I'm hoping you are going to tell me that you have no clue as to the intent of the programmer. The intent could have been to compare the two strings in a case sensitive manner, ignoring cultural rules (which is what actually is happening in the above snippet). It also could have been to compare object equivalence, just that the programmer forgot to cast the strings to object first. The intention could also have been a case-insensitive comparison. Another possible intent was to compare strings using cultural rules. The intent of the programmer simply is not clear!

Rewriting the comparison line in the above snippet correctly, assuming the actual behavior of the code was the intent should look like this:

if ( String.Compare(a, b, false,
System.Globalization.CultureInfo.InvariantCulture ) == 0 )

This tells me that the intent of the programmer was to compare the two strings in a case sensitive manner, ignoring cultural rules . There is no other possible intent given the above line of code. This is not saying that the code is correct - just that the intent of the code is clear to those lucky enough to read it, and that the developer thought it through.

If I am skimming through some code and see a non-String.Compare() string comparison, I will sprinkle a //BUGBUG: comment above it. Why? The simple string comparison syntax has been a rather common cause of bugs in C# code - much like the C/C++ switch statement. When investigating an issue in a piece of code, I will first look for these BUGBUG's and the TODO's as a starting point - often with better-than-random chance results.

Custom Action Tutorial Part III – What we did in Part II

This is the third in a series of articles about building Custom Actions. If you are coming in late to the party, check out Part I and Part II first. If I tried to explain what we were doing and why in the last installment of this tutorial, you would be even more lost now than you probably are. This article is an attempt to explain exactly what it was we did in Part II and why. Let’s start by going over each project in our solution.

The HelloWorld project

We needed something to install. This happens to be it. I could have picked notepad.exe or something, but it is nicer to see how Visual Studio could grab project outputs and put them into an MSI fairly easily.

The CustomAction project

This one is really the meat of what we are trying to learn. The other projects are sort of extraneous if you are using a tool other than Visual Studio to build your MSI. Microsoft did a pretty good job of explaining why you may want to use Custom Actions in the first place, so I’m not going to rehash that.

We created a Custom Action DLL. The reason I chose a DLL is because it is the most flexible means of tightly integrating a Custom Action with the installation engine. Additionally, there is not much you cannot do in a Win32 C++ DLL. An executable Custom Action cannot interact with the MSI engine. A VBScript or JScript action does have this capability, but if you are not careful, these are not bulletproof, and usually won't allow you to do much anyway.

We added a "Module Definition File" (.def) file to the project. This .def file is read by the linker and is used (for our purposes, anyway) to prevent mangling of function names (to be technically correct, specify the calling convention to be used). Mangling is entirely normal when a DLL is created without using the extern "c" function declaration – in fact, it is called by the more pleasant name "Decorating" the function names. If you run "Dependency Walker" (depends.exe in the Bin folder of the Platform SDK), and open a DLL in it, the listing of function exports appears. Pick FrameDyD.Dll from the same Bin directory and look at it. You will notice some ? and @ symbols around the names of functions. The .def file allows us to not create these mangled exports – you can verify this by opening the custom action dll we built in our project and see this for yourself. Ultimately, this makes the process less prone to error when entering data in the custom action table of the MSI. When adding the Custom Action via Visual Studio, it will handle the mangling automatically, so for this specific project this was not strictly necessary. If you are adding this DLL using another installation designer, you will thank me for not mangling the function names. Anyway, the .def file must contain the function names of any function you wish to make callable from outside the DLL. In our case, we wanted all four of the functions we added to be callable by the MSI engine, which is why they were added to the DEF file. There are tons of other ways to accomplish this goal

The function signatures we added to the CustomAction.cpp file are based on the requirements of the DLL Custom Action type. We return an unsigned integer (UINT) and accept a single parameter that is the handle to the installation (MSIHANDLE). We must use this handle to interact with the MSI Engine. The UINT return type indicates our status to the MSI engine.

The file "msi.lib" is the link library for the MSI engine. All custom action DLL’s must link to this file in order to call any MSI functions. The instructions in Part I explained an alternate way to inform the linker what to link with. One way is not necessarily better or worse than the other. The functions defined in the header files msi.h and msiquery.h (that we included in the stdafx.h file) all require linking to this library.

We made additional solution configurations to give us a UNICODE project type. While this is not intended to be a primer on UNICODE, all NT kernel OS’s are UNICODE under the hood. Windows 95 through ME are not UNICODE, although they can support some UNICODE extensions with the Unicode Library. For dll's and executables that only will run under Windows NT, 2000, XP, or 2003, the only option you should use is UNICODE. Therefore, if you are targeting only NT based systems, build and use the UNICODE projects. If you are supporting Windows 9x based machines use the non Unicode versions. To make the Unicode transition easier, Microsoft provided the tchar.h header that we include in the stdafx.h file that deals with strings based on compiler settings (actually preprocessor macros), so you may write code that will compile and work as native UNICODE or ANSI based on compiler settings. The TEXT() macro used in the "Hello MSI!" exercise is an example of one such tchar.h macro. Future Custom Action examples presented here will use the tchar.h macros when the compatibility of the Custom Action spans both Windows 9x and NT based platforms. Actions targeting Windows NT based systems will only compile in UNICODE, and will not use the tchar.h macros.

The final thing we added to the CustomAction project was the header include for strsafe.h. This header file describes the safe string functions intended to replace the C/C++ standard library string functions as well as specific Windows string handling implementations. These functions always start with the word "String" followed by the type of count provided as a parameter to the function, either the "Cb" which is a count of bytes, or "Cch" which is a count of characters. Remember that Unicode implementations use 2 bytes per character as opposed to the ANSI 1 byte per character – so the Cb and Cch designations are important. The last important fact about this header is by default it will deprecate the unsafe string handling functions – if you use sprintf(), you will get a compiler warning.

The Setup1 Project

Hopefully we are pretty clear on the basics for the first part of this one – an EXE is generated, Visual Studio generates a basic MSI and adds entries in the Feature, Component, File, Shortcut, and other tables to install the HelloWorld exe and the shortcut for it.

When we added the Custom Actions, we did a few things that affected the compiled MSI. Use Orca (which you installed in Part I) to view the generated MSI and follow along - try to remember back to the Custom Action theory presented in Part I:

1. Added a new Component and File table entry that installs the CustomAction.dll file.
2. Added four entries to the CustomAction table, referencing the DLL that we installed with the product. Notice the Source column of the CustomAction table is a reference to the key of File table for our Custom Action DLL. The CustomAction table's Target column is the DLL entry point (remember this is nothing more magical than the function names in our .def file). The action name is just a randomly generated GUID. The type column starts its life as a Type 17 which indicates msidbCustomActionTypeDll + msidbCustomActionTypeSourceFile, and to this number is added the following:

   Action Type	Decimal Value	Description
   Uninstall	1041	= 17 + msidbCustomActionTypeInScript
   Install	1041	= 17 + msidbCustomActionTypeInScript
   Rollback	1297	= 17 + msidbCustomActionTypeInScript + msidbCustomActionTypeRollback
   Commit	1553	= 17 + msidbCustomActionTypeInScript + msidbCustomActionTypeCommit

3. Looking at the InstallExecuteSequence table, note that all but the uninstall custom action is scheduled before the RegisterUser Standard action, and the uninstall custom action is scheduled before the UnpublishComponents action. Note that these actions are conditionalized on the action state of the component, denoted by the dollar-sign - you can see this clearly in the InstallExecuteSequence table. In a nutshell, this condition will cause the custom actions to run only when the component it is tied to (the Custom Action DLL) is being added or removed from the system.

Visual Studio does not give us a bunch of options when dealing with Custom Actions, so we are pretty much stuck with what they offer – which is the ability to add an additional condition (other than the component install state), stuff what goes into the CustomActionData property fairly easy, and sequence the custom actions relative to each other. You cannot change sequencing of the custom action (like making it occur relative to another standard action other than RegisterUser or UnpublishComponents, embedding the custom action in the binary table, or add immediate custom actions. Other authoring tools offer the ability to be tons more flexible, but we are stuck with these limitations for now.

That concludes Part III of the Custom Action Tutorial. In the next part, we will learn how to debug the Custom Action.

Custom Action Tutorial Part II – Creating the Project

[Update 3/10/2009: Updated download link. Thanks for hosting, Aaron!]

In Part I we covered the dry material - what custom actions are, and how to use them. In this part of the tutorial, we will configure the development environment and create a baseline project to start from when developing future Custom Actions. Please make sure your system is configured as described in the first part before continuing.

I am providing a download link for the final project described in this part for convenience (see the end of this post for the link), but I recommend first time custom action developers follow the steps so they understand what is going on. For my more experienced readers, please bear with the simplicity here, but review the project settings before downloading or using the template in future parts.

Create a Basic Installation Project

To create our solution, open Visual Studio 2003. We are going to create two projects to start with – one a Win32 Project called "HelloWorld", and a deployment project that will create an MSI to install it. Go to the File->New->Project menu. Select the Visual C++ Projects->Win32->Win32 Project. Call the name of the project "Hello World" and place it into a directory such as C:\Projects\CATutorial. Select Finish in the Wizard dialog. When the IDE initializes, you should see the Solution Explorer window – Right-click on the Solution 'HelloWorld' line at the top of this box and select Add->New Project. Find the Setup and Deployment Projects category and select Setup Project, and select the Setup Project Wizard – you can accept the default name of Setup1. Hit next, and accept the defaults for page 2. On page 3, check the Primary Output from HelloWorld option and hit Finish. When the project opens, select the Users Programs Menu and right-click in the pane to the right. Select Create New Shortcut. Double-click Application Folder and double-click the Primary Output from HelloWorld (Active). Rename the shortcut if you wish.

Now we have both an EXE and an MSI project that installs the EXE plus a shortcut to it. At this point, we can build and test our MSI by right-clicking the Setup1 project and selecting Build. Next, right-click the project and select Install. You will be walked through the standard MSI project dialog sequence – just accept the defaults and finish the install. A shortcut is installed to Program Files in the Start Menu, which will open a window that pretty much does nothing. If you get this far, you are on the right track.

Add a DLL Project

To make a Custom Action DLL, we want to now add a new project – a Win32 Project that is a DLL. Follow the add project instructions for "HelloWorld" above, but call it "CustomAction" and in the Wizard, select the Application Settings and pick DLL for the Application Type. Press Finish. We now need to add some items to this project – please follow this closely.

1. Right-click on the CustomAction project and select Add->New Item. Find the Module-Definition File (.def) and select it. Call the file CustomAction and press Open. We'll add some content to it in a later step.
2. Right-click on the CustomAction project and select properties.
3. In the Configuration dropdown, select All Configurations
4. Expand the Linker folder, and select input. In the Additional Dependencies edit box, enter msi.lib and hit Apply. NOTE: As an alternate to this and the previous 2 steps, you can use the following directive in the stdafx.h or any other source file to tell the linker to link with msi.lib: #pragma comment(lib, "msi.lib")
5. In the upper right hand corner of the Property Pages dialog box, click Configuration Manager.
6. Under the Active Solution Configuration dropdown, select New. Enter the name Release Unicode, select Release from the Copy Settings from dropdown, and leave the checkbox checked. Press OK.
7. Repeat the previous step, calling the Configuration Debug Unicode and select Debug under the Copy Settings From dropdown. Close the Configuration Manager.
8. We are now back at the CustomAction property pages. Select Debug Unicode from the Configuration dropdown. Under the General Configuration Properties, find the Character Set setting and select Use Unicode Character Set. Press Apply.
9. Repeat the previous step for Release Unicode. Press OK to close the CustomAction property pages.
10. Repeat the previous two steps for the "HelloWorld" project by opening the property pages for the HelloWorld project.
11. In the CustomAction project, under the "Header Files" folder, double-click the file stdafx.h file to edit it. Under the // TODO: line, add the following lines:
    #include <tchar.h>
    #include <msi.h>
    #include <msiquery.h>
    #include <strsafe.h>
12. It is always a good idea to add versioning information to all the compiler outputs you build, so right-click on the CustomAction project and select Add->Resource. Pick Version from the list by double clicking on it. Edit the information here if you so desire.
13. In the CustomAction.cpp file, add four functions based on the following template named Install, Commit, Rollback, and Uninstall – function names are case sensitive.
    extern "C" UINT __stdcall Install(MSIHANDLE hInstall)
    {
    return ERROR_SUCCESS;
    }
14. Edit the CustomAction.def file and make it look like this:
    LIBRARY CustomAction
    EXPORTS
    Install
    Commit
    Rollback
    Uninstall

As a final step, we want to add the Custom Action DLL to the setup project. Right-click on Setup1 and select View->Custom Actions. Right-click on Custom Actions in the pane that opens up and select Add Custom Action. Double-click on Application Folder, and then click Add Output. Select CustomAction from the Project dropdown and accept the default Primary Output and Active Configuration settings. Click OK, then click OK on the Select Item in Project dialog. You will notice that four custom actions are added, and in viewing the properties for each of them the EntryPoint corresponds to the function names we created in the CustomAction.cpp file.

At this point, lets right-click on the Setup1 project and build it. You will notice that the file "msi.dll" is detected as a dependency for the Setup1 project – as a matter of fact, it will create a warning during compilation. Right-click on the msi.dll file under dependencies and check the Exclude option. Rebuilding the Setup1 project should now generate no warnings.

If you have not uninstalled the setup project from the earlier test, do so now, and run the Setup1 installation. You should see no difference from the previous time you ran the setup. If you get errors, you may want to download the zip file that contains the projects and steps we have performed up to here, and use WinDiff from the Bin directory of the Platform SDK to see if you can figure out the issue by comparing the directories of my project against yours.

Save your workspace and exit Visual Studio. Create a .zip file of the project directories – if you used my directory structure, you want to compress the "C:\Projects\CATutorial" directory. We will use this as a template for any future C++ custom action tutorials.

Initially, I was going to end this part here, but I thought people would be cursing me, saying that this whole exercise was pointless and no fun! After all this work, we created a Custom Action that did absolutely nothing! Let’s rectify that! Open your solution and replace the Install function in the CustomAction.cpp file with the following code (Don't worry if you don't understand it, we will cover this later):

extern "C" UINT __stdcall Install(MSIHANDLE hInstall)
{
PMSIHANDLE hRecord = MsiCreateRecord(0);
MsiRecordSetString(hRecord, 0, TEXT("Hello MSI!"));
MsiProcessMessage(hInstall, INSTALLMESSAGE(INSTALLMESSAGE_USER + MB_OK), hRecord);
return ERROR_SUCCESS;
}

When you rebuild the project and install it, you will see a "Hello MSI!" message when the Install Custom action runs.

You can download the Visual Studio 2003 Project Template here:

(Special thanks to Aaron Stebner for hosting this for me). The next installment in this series will explain what we did (and why we did it)!

Custom Action Tutorial Part I – Custom Action Types and Sequences

This is the first part of a multi-part series on Custom Actions in the MSI world. The articles are designed to be read in order, as each one will build from knowledge gained in the previous ones. To give you a short roadmap, the series will start with some dry and boring theory, and the subsequent articles will start us down a path of writing Custom Actions in unmanaged C++, C#, and script.

Custom Actions are used when the power of the Windows Installer engine's Standard Actions are not enough to accomplish a given installation task. For instance, installing or controlling a service is trivial using MSI Standard Actions, but creating a virtual directory in IIS is not. To create that virtual directory, you will need to write a custom action. Other examples of installation tasks that require Custom Actions can be found here.

Before beginning, I'd like to make sure you have the proper environment to work under. I'm going to use Microsoft's Visual Studio 2003 as the environment of choice for this series. The writing level is intended to be a walkthrough for someone who may be academically familiar with C/C++. I am going to handwave much of the Win32 enumerated types and definitions - the reader who is completely unfamiliar with these topics can find additional resources in books, the MSDN library, or around the web.

This part covers some necessary background on the different Custom Action types and Custom Action scheduling. Mostly this is dry and boring theoretical stuff, but essential to understand. Hopefully, I summarized it fairly well and won't cause anyone to drift off to sleep during their lunch hour.

Setting up your environment

For this first part, you only need the Platform SDK and a free MSI editor called Orca (distributed with the Platform SDK) installed. Future parts will require the following setup, so prepare yourself by setting this up. The development environment should be set up as follows: Install Visual Studio 2003, the current MSDN Library, and the most current Platform SDK. The Platform SDK contains new and updated header files and link libraries to support the latest OS platforms, components, and service packs. The installation of the Platform SDK installs a shortcut Microsoft Platform SDK in the Program Files menu. Open this and find the option to register the platform SDK directories with Visual Studio and run it. Finally, in the installation directory for the SDK, there is a directory "bin" and inside this you will find orca.msi. I recommend installing Orca, as my instructions are based on this. You may also substitute the MSI editor of your choice, just be aware of the differences.

Let's get started

We are going to use Orca to investigate the Orca.msi file. So, find the Orca.msi file, right-click on it, and select Edit With Orca, and play along.

The Two "categories" of Custom Actions

There are only two major categories of custom actions. The category of a custom action is indicated by a number in the type column of the CustomAction table (You should be finding this table in Orca as you read along). This number is actually a bitmask - for folks new to programming, if you looked at the number in binary form, the presence of a 1 or 0 in a particular location is how all the flags and options are interpreted by the MSI engine. I will give the hex (or Hexidecimal) representations of these flags throughout (all hex digits begin with a 0x to indicate they are hex). You can use the Windows "calc" program in the scientific view to enter the hex and convert it to binary to see which flag toggles which bit. Using hex (or even binary) is perhaps the best way to reverse engineer the decimal value from the table into the flags it represents. The two major categories of custom actions are:

• Deferred - This is a modifier of (in hex) 0x400 or 0x4000 in the Type column of the CustomAction table, which corresponds to what MSDN calls (and you do not need to understand what these are yet) msidbCustomActionTypeInScript or msidbCustomActionTypeTSAware, respectively. What is important is Deferred actions are actions that modify the state of the system in some way. There are also some notable restrictions on deferred actions - you cannot change the value of a property, you can only read the value of a single property, and you cannot interact with the MSI tables. These can only be sequenced between the InstallInitialize and InstallFinalize actions in the Sequence table, lest you encounter the dreaded 2762 error "Cannot write script record. Transaction not started." error. You will find out the reasons behind this error later in this article. Additionally, there are options to specify a special kind of deferred action of "Rollback" or "Commit." These will also be covered later in this article.
• Immediate - This is a modifier of zero in the Type column of the CustomAction table - in other words it is any entry in the Type column that does not contain the bitmasks described in the Deferred section above. Immediate actions are used for UI tasks, evaluating the state of the system, or preparing data or the system itself for later modification from a Deferred action. Immediate actions should never modify the state of the system.

The Orca.msi I am looking at has two different custom action types - 0x23 and 0x33. Neither of these numbers has the 0x400 or 0x4000 bits set, so this must mean they are Immediate Actions.

The Type column of the CustomAction table

Custom Actions have a Type - this is the base number (or bitmask) that gets entered in the Type column. This Type column tells the MSI engine both what they do and how to interpret the Source and Target columns of the CustomAction table. The major Types are:

Action	Hex Code	Description / Notes
Call a DLL function	0x01, 0x11	The DLL function that is called must be specifically written for the MSI engine.
Run an EXE	0x02, 0x12, 0x22, 0x32	Any EXE will run. By default, if the EXE returns a code other than zero, the action will fail. There is a flag available to turn off this behavior.
Abort the Installation	0x13	Aborts installation and displays a Formatted message from the target column.
Set a directory	0x23	Uses formatted text from the Target column to set a Directory.
Set a property	0x33	Uses formatted text from the Target column to set a Property.
Run some Jscript	0x05,0x15,0x25,0x35	Bear in mind the normally present WScript object is not available in these scripts.
Run some VBScript	0x06,0x16,0x26,0x36	Bear in mind the normally present WScript object is not available in these scripts.
Run a nested installation	0x07, 0x17, 0x27	Nested installations are evil. Don't do it. Use a setup launcher or AppSearch to enforce setup order.

Hopefully you see the pattern here - all the EXE actions contain 0x2, all the JScript ones contain a 0x5, etc.

The Source Column of the CustomAction Table

The second hex digit from the right in the CustomAction Type column tells us how to interpret the Source column. This is a bit more of a stretch then above, but you should see the trend:

Hex Code	Description / Notes
0x00	The Source column is a key into the Binary Table, where the the file needed to run this action is stored.
0x10	The file needed for this action is installed as part of this installation, the Source column is the key into the File table. This fact places restrictions on where this type of action can be placed in the installation sequence. In the case of the nested installation, the Source column points to the msi path inside this msi's source tree. For the abort action, the source column is ignored.
0x20	Source column is the key into the Directory table for setting the directory or indicating the EXE's working directory,Source column is the ProductCode for the nested installations,Source is null for the VBScript and Jscript actions. Note that you cannot return anything other than a success code with actions of this type. Actual script is stored in the Target column of the CustomAction table.
0x30	Source is the Property name where the value of the property will be set by the set property action, Source column contains the script code for script actions, or for the EXE action the Source column is the property that contains the path of the EXE to run.

The Target column of the CustomAction table

The Target column is dependent on the major type of the CustomAction, reading the MSDN Summary List of All Custom Action Types will explain this column in more appropriate detail. Looking back to the Orca.msi file - the actions listed translate to the following:

• 0x23 - Set the directory listed in the Source column to the Formatted Text in the Target column.
• 0x33 - Set the property listed in the Source column to the Formatted Text in the Target column.

How and When is a Custom Action run?
Now that we know a Custom Action is either Immediate or Deferred, and can be or do just about anything, we need to answer the question "How does a Custom Action get fired off?" Entering a Custom Action into the CustomAction table really only makes it available to be called - so just adding a line in the CustomAction table effectively does nothing. There are three methods of firing off a Custom Action:

• In the sequence tables. This is probably the most common way of running a custom action, and the focal point of our discussion for the remainder of this article series.
  
• In the ControlEvent tables. Essentially this is the method to launch a Custom Action in response to a UI event such as a button click. On Windows Server 2003 or better, custom actions run this way cannot send messages with MsiProcessMessage() calls, however Session.Message() calls from the automation interface work just fine (technically, prior to Windows Server 2003 neither message API officially works). Obviously, the installation must be running in full UI mode to fire these off. The Custom Action type launched by this method must be an Immediate type - remember we should never alter system state in the UI phase. I'm not going to cover how to run an Immediate Action from a dialog here using ControlEvent, perhaps in a later article.
  
• By a call to Session.DoAction() or MsiDoAction(). This is a way of calling one Custom Action from another. Although this won't make sense until it is explained later on, you can not call a Deferred Custom Action in this manner unless it is called while the "installation script" is being written - but it should be OK to call it from a custom action scheduled between InstallInitialize and InstallFinalize. More on the "installation script" concept later.

The Sequence tables

Before we can discuss the primary way Custom Actions are launched, you need a better understanding on how MSI sequences work. We will get into this in more detail later in this article so I am handwaving some important details here for the sake of getting a basic understanding. The typical installation is kicked off by double-clicking the MSI. This causes the installation engine to begin processing the InstallUISequence table (The processing of this table is skipped entirely if the UI mode is set to "basic" or "no UI" - both Full and Reduced UI modes actually do process this table). There are other sequences such as the AdminUISequence and AdvtUISequence that all follow the same logic for administrative and advertised installations. Hopefully you still have Orca open, so look at the tables as I describe them and follow along - sort the table by ascending sequence number.

The InstallUISequence table is processed in the order identified by the Sequence column (starting at sequence number 1), executing each action listed(Standard, Custom, and Dialog), as long as the Condition column for that action evaluates to true or is null (empty). The negative values in this table are "jumped" to on termination of the setup based on the result code of the installation. Once the "ExecuteAction" action is encountered, the InstallExecuteSequence is run. When the InstallExecuteSequence is complete, control will return to process the remaining UISequence table or it will jump to the corresponding negatively numbered sequence based on the result code of the installation process. Every action in the *UISequence tables are run in the currently logged on user's process for security reasons, and this is known throughout MSDN's documentation as the "client" portion of the installation.

The InstallExecuteSequence is where things get interesting, and differs based on platform.

• On Windows NT based systems, this table is evaluated and run in a separate process from that of the User Interface. See my earlier article on Properties for information as to what is passed from the "client" (UI portion) to the "server" or "service" portion of the installation. To recap, the admin user has two processes owned by him/her, one is the UI sequence processing, and the second is the Execute sequence processing. The non admin user installing with elevated privlidges can have three processes - the same two as above, plus a third process, owned by the System, which executes the deferred
  msidbCustomActionTypeNoImpersonate actions.
• On Windows 9x based systems, both the *UISequence and *ExecuteSequence and all actions contained therein are run in the same process.

Actions in the *ExecuteSequence should only use UI interactions that use the Session.Message() or MsiProcessMessage() API's and not reference the Dialog table or contain UI's (such as MessageBoxes, status bars, shell progress indicators, dialogs from 3rd party libraries, etc.) in order to respect the UILevel wishes of the user.

Now or Later

Let's continue the discussion by distinguishing the Immediate vs. Deferred custom action categories a bit more. An immediate action is one that is "run" or "executed" immediately when the action is encountered in the sequence. A Deferred action is one that, when encountered in one of the Sequence Tables, is written to a sequential "installation script" with some metadata - you can think of this as a "To-Do list". Wherever you see the words "script" or "written to the script" in MSDN documentation, this is what I am referring to. More on deferred actions and scripts later - let's get the Immediate Custom Actions out of the way first.

Immediate actions have a method of assuring they will only be executed a set number of times - this is perhaps the most confusing page of documentation in the MSDN library. The Custom Action Type msidbCustomActionTypeFirstSequence will cause the action to not run in the Execute sequence if it already ran in the User Interface sequence. The msidbCustomActionTypeOncePerProcess flag is similar to the msidbCustomActionTypeFirstSequence flag, with the exception that it will run an action again in the Execute sequence if the Execute Sequence is being run in a different process than the User Interface process (In English - it will run in both sequences on NT machines, and only once on 9x based machines). The msidbCustomActionTypeClientRepeat flag means if you run the MSI in silent mode (and therefore don't have the UI sequence table processed) , the Custom Action wouldn't execute, but if you ran it in full UI mode it would execute. The lack of any of these flags causes the action to be run always. Remember that Immediate Actions should not modify the state of the system in any way. Deferred actions cannot have the properties described in this paragraph. Deferred Actions are "recorded" in what is called the "installation script" before anything is actually run/executed. The script is created in the *ExecuteSequence when the InstallInitialize action is encountered, therefore deferred actions can only be sequenced after this action. The only time an item is written to the script is if the Condition is true at the time it is to be written to the script (as indicated in the Condition column of the Sequence table). In reality, there are two scripts being generated, the "installation script" and the "rollback script". I'm going to treat it as if there is only one script for simplified understanding.

Deferred Actions and the script

Think of this example: The MSI Engine is going through each of the actions in the InstallExecuteSequence in order of the sequence numbers, when - blammo - it hits the InstallInitialize action. At this point, it creates a data structure known as the "script" and starts recording all actions that meet the condition specified in the Condition column and are marked as msidbCustomActionTypeInScript along with some magical metadata (In other words it only records deferred actions in the script). If an immediate action with a proper condition (that also is compliant with the multiple execution flag setting) is encountered, it runs immediately. Otherwise, it keeps adding stuff to the script until it hits an InstallExecute, InstallExecuteAgain, or InstallFinalize action is encountered. Ignoring InstallExecute and InstallExecuteAgain for now, when InstallFinalize is encountered, it completes writing to the script and seals it from further additions. Then it begins processing the script by executing the actions, top down. The details of how the list is processed and the effect of InstallExecute and InstallExecuteAgain is covered in a bit.

So, what exactly is this magical metadata mentioned above? We already know that the condition column was evaluated before the decision was made to write the action to the "script" - if the condition was false, the task was not written. As near as I can figure, this metadata holds three major things:

• The value of a property that having the same name as the CustomAction - the so-called CustomActionData property you have undoubtably heard so much about. Additionally, the ProductCode and UserSID properties are available in a Deferred action.
• How to Run it flags (derived from the Type column of the CustomAction table)
• The type of action (DLL, EXE, Script, etc.) and the path to the CustomAction code.
  
• Impersonation type - The flag msidbCustomActionTypeNoImpersonate will cause the action to run in the system context vs. the otherwise usual currently logged-on user context. The exception to this rule is a server with Terminal Services which normally runs actions in the system context unless the msidbCustomActionTypeTSAware flag is set (flag valid on Windows 2003 Terminal Server only).
  
• msidbCustomActionTypeAsync indicates that once the action is kicked off there is no need for the MSI engine to wait for it to complete before executing the next action. However, the MSI engine does want to know the return code, and as such will wait around at the end of the sequence until it finally returns if the action takes that long. All actions are synchronous unless this flag is set. Only EXE type custom actions can also combine the msidbCustomActionTypeContinue flag so that the installation can exit even while the Custom Action EXE is still running asynchronously.
  
• msidbCustomActionTypeContinue indicates that the exit code of the custom action is ignored - therefore the sequence will progress even if the return code indicates failure.


• When to run it flags indicates three possible things (again derived from the Type column of the CustomAction table). As briefly mentioned earlier there are actually two different scripts written - Commit and Rollback actions are both written to a separate "Rollback Script", but ignore this for now.
• "Normal Deferred" assuming nothing has failed and the list is being processed top to bottom - msidbCustomActionTypeTSAware or msidbCustomActionTypeInScript
  options without msidbCustomActionTypeRollback or msidbCustomActionTypeCommit flags
• "Commit" - Run after the entire (sealed) script is processed once successfully (After InstallFinalize is encountered and the complete script has been processed once successfully). Actions in this category have themsidbCustomActionTypeTSAware or msidbCustomActionTypeInScript combined with msidbCustomActionTypeCommit flags in the Type column. This is only written if Rollback is not disabled - more on this later..
  
• "Rollback" - if another deferred action failed, this will be run. This is only written for the msidbCustomActionTypeTSAware or msidbCustomActionTypeInScript
  when combined with msidbCustomActionTypeRollback actions, and only if Rollback is not disabled - more on this later..

How the Script is Processed

To recap, we have a "script" containing deferred actions along with some metadata that we started collecting when InstallInitialize was encountered in the sequence. We can potentially start processing the list before it is completely written (assuming a InstallExecute or InstallExecuteAgain was encountered prior to InstallFinalize in the sequence tables).

Processing always begins top-down, and there are up to three "phases" in the script processing based on the "when to run it" metadata described above.

First, only "normal deferred" actions are processed. Each deferred action is executed only once - if an InstallExecute or InstallExecuteAgain action was encountered in the sequence tables causing a portion of the script to be executed prior to the script being completely written, "normal deferred" actions already executed are skipped.

If an error is thrown during the execution of a "normal deferred" action, the processing of the list reverses and starts moving backwards from where it currently is processing, executing the "On Rollback" actions in the list. Rollback can be "turned off" at a system level, if this is the case, Rollback actions will not be written to the script, and will not be executed.

Assuming all "normal deferred" actions are successful, processing of the script begins executing at the top again, this time executing the "On Commit" actions. Retuening a failure from a Commit Custom Action will cause any previously written Rollback actions to be executed. Because this causes some interesting and practically impossible to test scenarios, it is important to write solid Commit actions that do not fail or also specify the msidbCustomActionTypeContinue flag in the CustomAction table. Rollback can be "turned off" at a system level, if this is the case, Commit actions will also not be written to the script, and will not be executed.

Dealing with Rollback and Commit

If you are familiar with database concepts, Windows Installer is transactional. In other words, a failure to install a MSI package should leave the system in the exact state it was prior to beginning the installation. As such, Rollback and Commit actions are a part of the lexicon of Windows Installer, and understanding this when writing a Custom Action is essential. Also essential is understanding that Rollback can be disabled on a system level, thus, Rollback and Commit actions are not executed.

To adhere to the Windows Installer "Best Practices," all changes that are made to a system are in a "deferred" action - as an example, the "InstallFiles" standard action is actually a deferred action under the hood. Although not apparent in this manner (nor is apparrant by looking at the Sequence tables), the InstallFiles "normal deferred" action creates a backup of all the files it replaces if Rollback is enabled. This way, if an error occurs, all the replaced files can be restored to their previous state if an error occurs in a later deferred action through the Rollback process. The Commit actions are only run after all the "normal deferred" actions are run successfully - and the MSI engine uses the Commit action to delete the Rollback files.

Because of their nature, Rollback and Commit actions cannot be asynchronous.

A "normal deferred" Custom Action should only save rollback information if the RollbackDisabled property is not set. If it is set, Rollback and Commit actions will not be run, and this will cause your saved rollback data to remain after the installation is completed. You will need to pass this data into your "normal deferred" CustomActionData property in order to check if you should save rollback information.

Other Considerations

Inside a CustomAction, you may need to evaluate if a patch or repair is occurring. Since deferred CustomActions do not have the ability to check properties (other than the few properties mentioned earlier), you need to use an Immediate action to check (and pass on to the deferred action via CustomActionData) the value(s) of REINSTALL, PATCH, and/or MsiPatchRemovalList properties. For more details, see the Windows Installer Team Blog's comments on the subject or Heath Stewart's post.

To determine if you should save Rollback information in your deferred action, you will need to do the same technique as above to pass the value of the RollbackDisabled property. Other properties of interest for deferred CustomActions (depending on what they do) is ALLUSERS, and possibly UILevel.

To handle user requests to cancel the installation in any custom action, make sure that you check the return value from MsiProcessMessage() (or Session.Message()) calls to handle the IDCANCEL return value. I will be showing examples of handling progress messages in future posts on this topic. In the meantime check out the Windows Installer Team blog entry on the subject.

Another tricky concept is scheduling a reboot, if required, from a deferred Custom Action. Normally, you can't! To work around this oversight you have several options. The one I use is creating a file on the filesystem that the user running the installation can have delete access to. This is where the ProductCode and UserSID properties available to a CustomAction come in handy. Find the temp directory of the user using their SID and write a file with the name "ProductCode_rebootme" there. In an immediate action scheduled after InstallFinalize, check for the existance of this file and if present, delete it and call MsiSetMode() to indicate a reboot is required.

MSDN provides a few pointers on security here and here. In my opinion, it is extremely difficult to author a "data secure" MSI. If you need to pass internally sensitive data to a deferred custom action, use encryption on the CustomActionData in addition to adding sensitive properities (and the name of the deferred custom action) to the MsiHiddenProperties and using the msidbCustomActionTypeHideTarget CustomAction type flag. It is possible to digitally sign an installer package and its cabinet files. It is trivial to circumvent these measures if someone wanted to - and in a typical administrative installation, the msi signing is removed anyway. Think about data security in an MSI as glass in a jewelry store - it only keeps the honest people honest.

If a Custom Action needs to use disk space (to install a database, for example), you need to author (or modify at run time) the ReserveCost table. This is keyed on the installation state of a component.

The use of COM in a CustomAction is fine - when a dll-based Custom Action runs, it is created on its own thread. There is some conflicting documentation on this, however. MSDN claims CoInitialize is called in a per-machine installation and is not called in a per-user installation. A post here (scroll through to the comments) indicates that CoInitialize and/or its brethren CoInitializeEx is not called on this thread. I like MSDN's advice to not quit if it is determined the thread is already COM initialized.

Terminal Server is special. Throughout this article I have mentioned the *TSAware flags. In a per-machine installation on a Terminal Server, actions not marked with this flag are run as LocalSystem, where on a non-terminal server system they will normally impersonate the calling user.

The way patching and upgrading is handled, and its effect on Custom Actions, is the subject of a future article in this series.

The Ten Cent Summary

Custom Actions can be very powerful, and can be run in a variety of different ways and at a variety of times. The important things to remember when deciding on how to design them are:

• Immediate actions are normally used to "set up" deferred actions (by stuffing CustomActionData) or to simply evaluate the state of a machine. Immediate actions are NEVER to be used to alter the state of a machine.
• Deferred actions come in three different flavors, and where you have one, you should have all of them - the "normal deferred", rollback, and commit. The "normal deferred" should save undo information on the system somewhere if rollback is enabled. The rollback action will use this undo information to restore the initial state of the machine. The commit actions will clean up the rollback's undo information. Deferred actions can only be included in the execute sequences between InstallInitialize and InstallFinalize.
• Rollback only occurs if an error is encountered while processing deferred actions (including Commit actions) between InstallInitialize and InstallFinalize in the *Execute sequences.
• Because the conditions of deferred actions (including commit and rollback actions) are evaluated at the time the script is written, the same condition should be used for the sister actions.
• Actions should be written to run correctly regardless of the UI mode and respect the wishes of the user when it comes to UI levels.
• Never use a nested installation custom action type.
• Understand the flags and options for scheduling and running actions as presented above.

Part II of this series will cover writing our first Custom Action in C++!