Foo Theory

Workflow Services Primer

What are WF Runtime Services?

WF is a foundation framework which provides a number of components that together provide the necessary infrastructure to execute a workflow instance. When you take a look at the overall architecture of WF, you notice that it has been designed to be very extensible and configurable. A high level WF architecture diagram is provided below:

In this article, we want to talk about the box in blue, which represents the Workflow runtime services. Runtime Services are class instances that you create and register with the Workflow Runtime during application startup (typically in the host startup code). Each service has a particular purpose within the Workflow system. Workflow Services come in two varieties:

1. Core Services.
2. Local Services.

WF Core Services

The behavior and functionality of the WF core services has been pre-defined by Microsoft. In some cases, the Workflow Runtime will register its own default implementation of a core service if you don't provide your own (a typical example is the default Scheduler Service). In other cases, the Workflow Core Service is optional and a default instance is not automatically registered for you, such as is the case for the Workflow Persistence Service.

WF Local Services

WF Local Services are developed by you to serve any of your current needs. One common use of local services are Local Communication Services which serve a communications conduit to and from your workflow instances. Once you have created such a Local Communication
Service class, you can register an instance of it with the External Data Exchange Service, which is one of the Core Services mentioned earlier.

The purpose, design and implementation of a WF Local Service is completely up to you. You register a Local Service in exactly the same way that you register a core WF service. Once your Local Service is registered with the runtime, a workflow instance can invoke methods on the service, or handle events that originated from the service. In this context, local service are an important mechanism used by workflow instances to communicate with other parts of the same or other workflow instances, or with the hosting application.

WF Core Runtime Services Examples

A number of out-of-the-box core runtime services provided by the Microsoft Workflow Foundation. As was mentioned earlier, some of these services are required, and the Workflow Runtime will register a default instance for you if you don't explicitly register your own instance. Other services are completely optional and can be registered on an as-needed basis. 

A sample of some of the Core Runtime Services provided by the Workflow Workflow Foundation are shown below:

• Persistence Services. The Persistence service saves and restores the current state of the workflow instance to a durable medium. Out of the box, WF provides the SqlWorkflowPersistenceService class which supports  SQL Server 2000, 2005 or a SQL Express instance as the durable medium. This service is an optional Core Workflow Runtime Service
• Tracking Services. A Tracking Service is designed to monitor the execution of a Workflow Instance through the the recording of tracking events for the running Workflow Instance. Events can be collected at various levels:
  1. At the Workflow Level. At this level a tracking service can collect events such as the start and stop of a workflow instance, the loading and unloading of an instance, when a workflow was persisted using a persistence service etc.
  2. At the Activity Level. For an individual activity, a Tracking Service can record when the activity was initialized, when it started executing, when the activity reached the closed state, together with more exceptional situations such as whether or not the activity was canceled, reached a faulting state or needed to be compensated within the scope of a compensation block.
  3. At any Custom Level. The tracking infrastructure allows the programmer to defined custom data collection information points, and when this information should be collected, based upon the Tracking Profile (see next).

Tracking services collect their information based upon a Tracking Profile. A default Tracking Profile is provided, but the workflow designer can create a Custom Tracking Profile, which combines exactly those events that are of interest to the particular type of workflow.

Again the Tracking service is an optional Core Workflow Runtime service, and also, unlike other core workflow services, you are not limited to a single tracking service at any one time. You can add multiple tracking services to the Workflow Runtime at startup and each service will have the opportunity to work with and record the same tracking data. Out of the box, Windows Workflow Foundation provides the SqlTrackingService class, which supports a SQL Service instance as it's storage medium.

• Scheduling Services. The Workflow Scheduling Services create and manage the threads used by the runtime engine to execute a Workflow Instance, so the Windows Workflow scheduling services manage how workflow instances are scheduled onto threads for execution by the Workflow Runtime engine. Two different types of scheduling services provided by WF are:

1. The DefaultWorkflowSchedulerService. This is the default scheduling service provided by the runtime. This service will schedule workflows to run on threads from the process-wide CLR thread pool. This is why workflows execute asynchronously on a background thread by default.
2. The ManualWorkflowSchedulerService. This scheduler runs the instances in a synchronous manner, in affect "borrowing" a donated thread from the host application to the Workflow runtime. This is the scheduler that you would typically want to use if you are hosting your workflow in ASP.NET. In this case, your request is already running on an ASP.NET thread, and you would not want to consume yet another CLR thread, in addition to which you potentially might have to synchronize this additional thread with you original ASP.NET thread. Therefore, it it much easier to leverage the ManualWorkScheduleService to allows the workflow to execute on your current ASP.NET thread.

Unlike the core Workflow Runtime Services that were mentioned before, the Scheduler Service is a required Core Runtime Service. I you do not specify a scheduler, the Workflow Runtime will register the Default Workflow Scheduler Services for you. You have the option to either imperatively or declaratively select the Manual Workflow Scheduler Server, or even a custom-developed scheduler service.

• Commit Work Batch Services. The Commit Batch Service manages the transactions used by the runtime engine maintain consistency between the internal workflow state and external data stores. When the workflow engine needs to commit a work batch, the runtime calls the current CommitWorkBatch service to perform this operation. Out of the box, the Workflow runtime provides two types of Commit Batch Services:
  1. The DefaultWorkflowCommitBatchService. This is the default commit service. Note that when you are using two or more transaction managers (for example both the Persistence and the Tracking service, running on different databases), then this class will have to resort to creating a distributed transaction (involving the DTS) to commit the transaction. Distributed transactions involving the DTS (or any type of other two-phase commit coordinator) will always have a negative impact on the overall performance of you application.
  2. When you batch of work involves only one resource manager (for example in the above example, you combined both the Persistence and the Tracking database into a single database), you can use the SharedConnectionWorkflowCommitBatchService Workflow Service. This service has the ability to perform commits on a batch of work involving a single resource manager without requiring a distributed transaction, and can therefore perform better under this specific scenario.

Creating a Local Workflow Runtime Service

Scenario Description

My need to create a custom Local Workflow Runtime Service originated when I was creating the SMO Custom Activity Library. This library contains activities which can backup and restore a database. As you know, backing up or restoring a database can be quite a lengthy operation, so it would be beneficial if the custom SMO activity had the ability to provide some type of feedback (such as a percent complete event) to the hosting application. The host could then use a UI mechanism, such as a progress bar to inform the user of the progress of the current operation.

Now, both the Backup and Restore classes in the Microsoft.SqlServer.Management.Smo namespace inherit from a common class called BackupRestoreBase. One of the events that this class can fire is the PercentComplete event. As a property of the class, you can set the PercentCompleteNotification value, which determines how often the PercentComplete event is fired. This is shown in the class diagram below:

If we had a way to register for this event, and the publish these events in a global manner back to any interested party (for example, our hosting application), the we would "be in business". Here is where a Local Workflow Service really comes in handy. To make these events globally available, we could following this approach:

1. First, we create a Local Workflow Runtime Service a service (let's call it our SMONotificationService), and register it with the Workflow Runtime at startup.
2. The host application creates an instance of a workflow, which contains an SMO Backup Custom Activity.
3. Upon execution, our Custom activity gets a reference to this service (using the ExecutionContext.GetService<ServiceType>() method).
4. The Custom activity passed the reference to the backup or restore activity (using a BackupRestoreBase reference) to the SmoNotificationService. This service can then register for the PercentComplete event for the backup or restore instance.
5. The SMONotification local service instance receives the local event, and converts it into a globally available event (in this case named BackupPercentComplete).
6. The local service publishes the BackupPercentComplete event, which can be subscribed to by any interested party, for example the hosting application, as shown below:

                // Create the SMO Notification Service, subscribe
                // to the BackupPercentComplete event
                SMONotificationService notificationService = 
                    new SMONotificationService();
                notificationService.BackupPercentComplete += UpdatePercent;  

Each of the above steps is outlined in the figure below:

So, our SMONotificationService is in fact playing the role of middleman, giving our Custom Activities to expose custom events to the world.

This is how a Custom Activity retrieves a reference to our SMONotificationService, and hooks up the PercentComplete event:

                // Create the backup instance, set its properties
                backup = new Backup();
                backup.Action = BackupActionType.Database;
                                .....

                // Hook-in our "percent complete" notification 
                // workflow service
                backup.PercentCompleteNotification = 10;
                SMONotificationService notificationService =
                    executionContext.GetService<SMONotificationService>();
                if (notificationService != null)
                {
                    notificationService.BackupOrRestore = backup;
                }

                // Start the backup
                backup.SqlBackup(Server);

The full source of the Notification Service is show below:

using System;
using System.Diagnostics;
using System.Workflow.Runtime.Hosting;
using Microsoft.SqlServer.Management.Smo;

namespace SMOUtilities
{
    /// <summary>
    /// This delegate describes the event that is sent out
    /// by the service if it receives a "Percent Complete"
    /// notification by the embedded SMO Backup instance
    /// </summary>
    /// <param name="percent"></param>
    public delegate void BackupPercentCompleteDelegate(
        object sender, SMOPercentCompleteEventArgs e);

    /// <summary>
    /// This class implements a notification service for
    /// SMO events
    /// </summary>
    public class SMONotificationService : WorkflowRuntimeService
    {
        #region private fields
        
        private BackupRestoreBase m_backupOrRestore;
        
        #endregion private fields

        #region public properties
        
        /// <summary>
        /// This property represents an SMO backup
        /// operation. When the caller sets this property,
        /// we register for "PercentComplete" notifications,
        /// enabling us to send out our own "PercentComplete"
        /// notifications to our Workflow Clients
        /// </summary>
        public BackupRestoreBase BackupOrRestore
        {
            get { return m_backupOrRestore; }
            set 
            {
                Debug.Assert(value != null);
                m_backupOrRestore = value;
                m_backupOrRestore.PercentComplete += backup_PercentComplete;
            }
        }
        
        #endregion public properties

        #region public events
        
        public BackupPercentCompleteDelegate BackupPercentComplete;

        #endregion public events

        #region constructors
        
        // Default constructor
        public SMONotificationService()
        {
        }

        #endregion constructors

        #region protected methods

        /// <summary>
        /// When our service is stopped, and we 
        /// contain a backup object, then we need
        /// to make sure that we unsubscribe for 
        /// the PercentComplete event
        /// </summary>
        protected override void OnStopped()
        {
            if (m_backupOrRestore != null)
            {
                m_backupOrRestore.PercentComplete -= backup_PercentComplete;
            }
        } // method OnStopped
        
        #endregion protected methods

        #region private methods

        private void backup_PercentComplete(
            object sender, PercentCompleteEventArgs e)
        {
            if (BackupPercentComplete != null)
            {
                BackupPercentComplete(
                    this, new SMOPercentCompleteEventArgs(e.Percent));
            }
        } // method backup_PercentComplete

        #endregion private methods

    } // class SMONotificationService
}

The custom activity can set a reference to its Backup or Restore instance through the BackupOrRestore property. As soon as this property is set, the service registers for the PercentComplete eventof the BackupOrRestore instance.

You can see that this local service contains a BackupPercentComplete event, which is fired each time a PercentComplete notification is received from its embedded BackupOrRestore instance. This BackupPercentComplete event can be subscribed to by anybody, including the hosting application.

Currrent Limitations

The current implementation was quickly written for demonstration purposes only. Therefore, the  SMONoticationService has a number of limitations:

1. It should be able to handle multiple concurrent instances of Backup or Restore activities. This could easily be done by maintaining a List of BackRestoreBase instances.
2. The BackupPercentComplete event needs to named more appropriately, and its corresponding Event Args class needs to be expanded.
3. If option (1) is implemented, appropriate locking mechanisms should be put in place, since multiple workflows can run on multiple CLR threads.

Conclusion

The main purpose of this article is the illustrate the versatility that is offered by Local Workflow Runtime Services. A well-designed local service can dramatically improve the power and flexibility of your application, while greatly simplifying your code