Collection: Exchange 2007 Common Query (4)

Troubleshooting Mail Flow Between Exchange 2003 or Exchange 2000 Servers and an Exchange 2007 Hub Transport Server

This topic provides information about how to troubleshoot mail flow between servers that are running Microsoft Exchange Server 2003 or Exchange 2000 Server and an Exchange Server 2007 Hub Transport server. This problem occurs when you have deployed at least one Exchange 2007 Hub Transport server in an existing Exchange organization. When you try to send messages between the two mail systems, the messages are put in the Unreachable queue on the Hub Transport server or in the Messages with an unreachable destination queue on an Exchange 2003 or Exchange 2000 server.

This problem can occur when no routing group connector is created during the installation of the first Hub Transport server in an existing organization. A routing group connector is required for mail flow between Exchange 2007 and Exchange 2003 or Exchange 2000. During setup, two routing group connectors are automatically created to enable mail flow in both directions. If you use the Setup wizard, select an Exchange 2003 or Exchange 2000 bridgehead server to act as the source server for one routing group connector and as the target server for the reciprocal connector. If you use Setup.com to perform an unattended installation, you must provide the /legacyroutingserver parameter to automatically create the required routing group connectors. If you do not perform this procedure during setup of the first Hub Transport server, the two e-mail systems cannot determine a route between them. Messages that are sent from users with mailboxes located on Exchange 2003 or Exchange 2000 servers to recipients with mailboxes located on Exchange 2007 Mailbox servers are put in the Messages with an unreachable destination queue on an Exchange 2003 or Exchange 2000 server. Messages that are sent from users with mailboxes located on Exchange 2007 Mailbox servers to recipients with mailboxes located on Exchange 2003 or Exchange 2000 servers will queue in the Unreachable queue on the Exchange 2007 Hub Transport server. 

The Unreachable queue is a persistent queue that contains messages that cannot be routed to their destinations. Microsoft Exchange can resolve and locate the recipient. But Microsoft Exchange can't determine how to reach the destination. The messages remain in the Unreachable queue until they expire or until the administrator resubmits the messages to the categorizer.

To resolve this problem, you must create the required routing group connectors. You can't use Exchange System Manager on the server that runs Exchange 2003 or Exchange 2000 to perform this procedure. You must use the Exchange Management Shell on the Exchange 2007 server.

 Procedure

 To use the Exchange Management Shell to create routing group connectors

1. Run the following command to create a routing group connector from Exchange 2007 to Exchange 2003 or Exchange 2000 and to automatically create the reciprocal routing group connector. This command sets the routing group connector cost as 10 and enables public folder referrals between the two versions of Exchange Server:

New-RoutingGroupConnector -Name "Interop RGC" -SourceTransportServers "Ex2007Hub1.contoso.com" -TargetTransportServers "Ex2003BH1.contoso.com" -Cost 10 -Bidirectional $true -PublicFolderReferralsEnabled $true

2. Run the following command to verify that the two routing group connectors are created:

Get-RoutingGroupConnector

Troubleshooting Mail Queues That Are Increasing on Edge Transport Servers

This topic explains how to resolve the problem of inbound or outbound e-mail messages that are stuck in queues on a server running Microsoft Exchange Server 2007 with the Edge Transport server role installed. When this situation occurs, you will typically see the following errors in the Last Error column in the Exchange Queue Viewer:

• 451 4.4.0 DNS Query Failed
• 400 4.4.7 Message Delayed

This problem commonly occurs as a result of a mistake in the configuration of the DNS settings of the Edge Transport server. Therefore, you can resolve this problem by correcting the DNS configuation.

 Before You Begin

Confirm that any firewall between your Hub Transport servers and your Edge Transport servers allow port 53 for DNS resolution and port 25 for SMTP traffic.

 Procedure

 To use the Exchange Management Console to reconfigure DNS settings when inbound mail is queued on an Edge Transport server

1. Start the Exchange Management Console on the Edge Transport server.
2. Click Toolbox.
3. Select Queue Viewer under the Mail flow tools category to open the Queue Viewer tool.
4. Review the information in the Last Error column. Note whether you have an inbound message queue for an accepted domain, such as "company.com", and if there is an error similar to "451 4.4.0 DNS Query Failed".
5. Verify the DNS configuration on the Edge Transport server as follows:
1. Log on locally to the Edge Transport server.
2. Open the Exchange Management Console.
3. Select the Edge Transport server in the Result pane, and then select Properties.
4. Select the Internal DNS Lookups tab.
6. The default configuration is All Available. Your Edge Transport server will need to do external and internal DNS lookups. You have two options available:
1. If you have multiple NIC adapters, and one is for the internal network, select that network card in Use network card DNS settings. The IP addresses will populate the box below with the DNS server IP addresses that are specified on the internal network card. Restart the Transport service, and then repeat step 5 to confirm that the configuration is correct. If you do not see any IP addresses, the NIC card may not be configured with DNS server entries. Populate the card with DNS settings, and then repeat step 5 to ensure that the settings are correct.
2. If you have only one network card, and it is using external public DNS, you do not want to change this setting because it will break external name resolution and e-mail flow. There are two options in this scenario. You can select Use these DNS servers and then select the IP address of the internal DNS server, or you can add a host file containing the DNS server information.
7. After making changes, test your DNS servers and name resolution with NSLOOKUP
8. Next, test ping and telnet to your internal mail server. If ping or telnet connections are failing, check to see if the Windows Firewall in Control Panel/ Services has been enabled. It is typically disabled. If it is enabled, it needs to be configured on the NIC cards to allow services for mail flow, such as SMTP, LDAP, the Edge Transport server LDAP ports, and testing protocols such as ICMP. Enable only those ports that are required for the services that you are using.

Troubleshooting Edge Transport Server Queues That Contain Mail Destined to a Hub Transport Server

This issue occurs when both Transport Layer Security (TLS) and Exchange Server Authentication are not configured on the default Receive connector of the receiving Exchange Hub Transport server. Therefore, you can resolve this issue by enabling TLS and Exchange Server Authentication on the default Receive connector.

 Procedure

 To use the Exchange Management Console to configure authentication to allow an Edge server to transfer mail to Exchange Hub Transport servers

1. Open the Exchange Management Console and navigate to Server Configuration\Hub Transport.
2. Select the default Receive connector for the Hub Transport server that you want to configure to receive mail from the Edge Transport server.
3. Right-click the default Receive connector, and then select Properties.
4. Select the Authentication tab.
5. Check the Transport Layer Security (TLS) check box.
6. Check the Exchange Server authentication check box.
7. Click Apply.
8. Click OK.
9. Force Edge Sync synchronization by following the steps detailed

Transitioning from Exchange 2000/2003 to Exchange Server 2007

A transition is the process in which you perform an upgrade to Exchange 2007, that is you move data from any legacy Exchange servers in your Exchange organization to new Exchange 2007 Servers, after which you decommission the legacy Exchange servers. A transition should not be confused with a migration, because unlike a transition a migration is the process in which you move data from a non-Exchange messaging system (such as GroupWise, Lotus Notes or SendMail) to an Exchange organization, or move data from a legacy Exchange organization in an existing Active Directory Forest to an Exchange organization in a new Active Directory Forest.

It’s important to note that unlike previous versions of Exchange, in-place upgrades from Exchange 2000 or 2003 to Exchange Server 2007 aren’t supported, because, among other reasons, Exchange 2007 is 64-bit and therefore requires the x64-bit version of Windows Server 2003.

Although in-place upgrades to Exchange 2007 are unsupported, I can assure you that a transition from Exchange 2000 or 2003 to Exchange 2007 in the same Active Directory Forest is a straightforward process, as I’ll show you throughout this article series.

Prerequisites

Before you even start thinking about deploying Exchange 2007 Servers in your existing environment, there are several requirements that must be fulfilled first.

You must make sure that the Exchange organization is set to Native Mode (no pre-Exchange 2000 servers) as shown in Figure 1.1 below.

                                      Figure 1.1: Exchange Organization set to Native Mode

Since any pre-Exchange 2000 servers that may exist in your Exchange organization must be decommissioned before you can switch to native mode, it means that any Exchange 5.5 Servers in your organization must be properly removed before you can perform this step. 'So does this mean that you cannot do a transition directly from Exchange 5.5 to Exchange 2007 in the same Active Directory Forest?' I hear some of you ask. Yes that is correct! Those, hopefully few, of you who still have an Exchange 5.5 organization who want to move to Exchange 2007 must first upgrade from Exchange 5.5 to 2000 or 2003 and then do the transition from Exchange 2000 or 2003 to Exchange 2007.

You must also make sure that any Exchange 2000 Servers in your Exchange organization run with Exchange 2000 Service Pack 3, and that any Exchange 2003 Servers have Service Pack 2 applied. In addition you should take note that if you plan to keep at least one Exchange 2000 or 2003 server in the Exchange organization, the following services are unsupported by Exchange Server 2007:

• Novell GroupWise connector (Exchange 2003 Service)
• Microsoft Mobile Information Server (Exchange 2000 Service)
• Instant Messaging service (Exchange 2000 Service)
• Exchange Chat Service (Exchange 2000 Service)
• Exchange 2000 Conferencing Server (Exchange 2000 Service)
• Key Management Service (Exchange 2000 Service)
• cc:Mail connector (Exchange 2000 Service)
• MS Mail connector (Exchange 2000 Service)

You must make sure that the Domain Controller that is the schema master in your Active Directory runs Windows Server 2003 with at least Service Pack 1 applied. This is also true for any Global Catalog servers in each Active Directory site in which you plan on deploying Exchange 2007. Actually I recommend you run Windows Server 2003 with at least Service Pack 1 applied on all Domain Controllers in the Active Directory Forest. This version supports Exchange 2007 service notifications, allows users to browse the address book in Microsoft Outlook Web Access and provides the ability to look up distribution list membership in a more efficient manner than in Windows 2000 Server.

Finally Exchange 2007 requires that the Active Directory functional level is set to Windows Server 2000 or Windows Server 2003 as shown in Figure 1.2 below. 

                                        Figure 1.2: Active Directory Domain Functional Level

If you’re unsure whether your Active Directory environment is ready for deploying the first Exchange 2007 Server, I recommend you run the latest version of the Exchange Best Practices Analyzer (ExBPA) to see if there’s anything you need to resolve before you continue.

                             Figure 1.3: Exchange 2007 Readiness Check Option in ExBPA

Preparing Active Directory

With all prerequisites fulfilled we can move on and prepare the Active Directory using the respective Exchange 2007 Setup.exe switches. Exchange 2007 Setup includes several switches; we’ll go through each of those related to preparing the Active Directory in this section.

Prepare Legacy Exchange Permissions

The first thing we need to do when deploying an Exchange 2007 into a legacy Exchange organization is to run Setup.com /PrepareLegacyExchangePermissions. This is in order to grant specific Exchange permissions in the Active directory domain(s) in which one or more Exchange 2000 or 2003 Servers exist, or where Exchange 2000 or 2003 DomainPrep has been executed. The reason why we must run the Setup.com /PrepareLegacyExchangePermissions is because the Exchange 2003 or Exchange 2000 Recipient Update Service otherwise won’t function correctly after the Active Directory schema has been updated with Exchange 2007 specific attributes.

In order to run Setup.com /PrepareLegacyExchangePermissions, you must open a Command Prompt window and navigate to the directory, network share or DVD media containing your Exchange 2007 Setup files, then simply type Setup.com /PrepareLegacyExchangePermissions and hit Enter as shown in Figure 1.4.

Figure 1.4: Running Setup.com with the /PrepareLegacyExchangePermissions Switch

Prepare Schema

The next command to run in order to prepare the environment is the Setup.com /PrepareSchema, which will connect to the Domain Controller schema master and import LDAP files to update the schema with Exchange 2007 specific attributes. In order to do so, open a Command Prompt window and type Setup.com /PrepareSchema followed by hitting Enter like we did with the previous switch. Setup will now update the schema as necessary as shown in Figure 1.5.

                                 Figure 1.5: Running Setup.com with the PrepareSchema Switch

Like was the case with the previous command, this can be done using the 32-bit version of Exchange 2007.

Prepare AD

The Setup.com /PrepareAD command is used to configure global Exchange objects in Active Directory, create the Exchange Universal Security Groups (USGs) in the root domain as well as prepare the current domain. The global objects reside under the Exchange organization container. In addition, this command creates the Exchange 2007 Administrative Group which is named Exchange Administrative Group (FYDIBOHF23SPDLT) as well as creates the Exchange 2007 Routing Group called Exchange Routing Group (DWBGZMFD01QNBJR).

As some of you may be aware, Exchange 2007 doesn’t make use of Routing Groups and Administrative Groups like Exchange 2000 or 2003 did. Administrative Groups have been dropped completely and message routing in Exchange 2007 is based on Active Directory Sites. But in order for Exchange 2007 to co-exist with Exchange 2000 or 2003, Exchange must create the mentioned Administrative Group and Routing Group, which can only be viewed via an Exchange 2000 or 2003 System Manager or by using ADSIEdit as can be seen in Figure 1.6 and 1.7 below.

Figure 1.6: Exchange 2007 Administrative and Routing Group in the Exchange 2003 System Manager

                     Figure 1.7: Exchange 2007 Administrative and Routing Groups in ADSIEdit

You can run the Setup.com /PrepareAD command before running /PrepareLegacyExchangePermissions and /PrepareSchema. Doing so will run the /PrepareLegacyExchangePermissions and /PrepareSchema commands automatically.

In order to run this command, open a Command Prompt window and type Setup.com /PrepareAD followed by Enter. Setup will now configure the organization as necessary as shown in Figure 1.8.

                                     Figure 1.8: Running Setup.com with the PrepareAD Switch

PrepareDomain and PrepareAllDomains

It’s also possible to prepare a local domain or all domains in the Active Directory using the Setup.com /PrepareDomain and Setup.com /PrepareAllDomains respectively. These switches will set permissions on the Domain container for the Exchange Servers, Exchange Organization Administrators, Authenticated Users, and Exchange Mailbox Administrators, create the Microsoft Exchange System Objects container if it does not exist, and set permissions on this container for the Exchange Servers, Exchange Organization Administrators, and Authenticated Users and create a new domain global group in the current domain called Exchange Install Domain Servers. In addition it will add the Exchange Install Domain Servers group to the Exchange Servers USG in the root domain.

As with the commands we have already been through, these commands also need to be run from a Command Prompt window as shown in Figure 1.9.

                             Figure 1.9: Running Setup.com with the PrepareDomain Switch

Installing Exchange Server 2007

We have reached part 2 in this 3 part article series covering how you transition from an Exchange 2000/2003 to an Exchange 2007 Server deployed in the same Active Directory Forest. For the purpose of this article we will only install one Exchange 2007 Server, and we’ll do so by selecting a typical installation of Exchange 2007. Since a typical installation of Exchange Server 2007 installs the Mailbox, Hub Transport and Client Access Server roles on the respective server, we must make sure the following software and Windows components are installed on the server prior to launching Exchange 2007 Setup.

Required Software

• Microsoft .NET Framework Version 2.0 (including this update)
• Microsoft Management Console (MMC) 3.0 (bear in mind MMC 3.0 is installed by default when using Windows Server 2003 R2)
• Windows PowerShell V1.0 (can be found here or on the Exchange 2007 DVD media)

Required Windows Components

Mailbox Server

• Enable network COM+ access
• Internet Information Services
• World Wide Web Service

When installing the Mailbox Server role, you also need to make sure you install the hotfixes mentioned in MS KB article 904639 and 918980.

Client Access Server

• World Wide Web Service
• Remote procedure call (RPC) over Hypertext Transfer Protocol (HTTP) Proxy Windows networking component (Required only if you are deploying clients that will use the Outlook Anywhere functionality, previously called RPC over HTTP)
• ASP.NET v2.0

Hub Transport Server

No additional Windows components are required by the Hub Transport server; however you must make sure that the SMTP and NNTP services are NOT installed.

Then Install Exchange Server 2007

Finalizing Deployment

Global Settings

Global Settings that have been configured on an Exchange 2000 or 2003 Server will be transferred to the Exchange 2007 Server automatically, as these settings are stored and read from Active Directory. This means that recipient policies, Internet Message Formats, SMTP connectors and Exchange delegation permissions are applied to user mailboxes stored on Exchange 2007 as well. Figure 2.7 below shows you the Default Policy which was originally created on our Exchange 2003 Server.

                        Figure 2.7: Default Policy in the Exchange 2007 Management Console

Also note that when the Exchange 2007 Server has been deployed in the legacy Exchange organization, any of the organization-level settings should be managed using Exchange 2007 Management tools (EMC or EMS) during the co-existence period.

Replicating Public Folders

When deploying an Exchange 2007 Server with the Mailbox Server role installed into a legacy Exchange organization, Exchange Setup will create one Mailbox database and one Public Folder database on the respective server by default as can be seen in Figure 3.1 below.

                               Figure 3.1: Exchange 2007 Mailbox and Public Folder Database

The Public Folder database is created so that you can replicate any Public Folder data stored on your legacy Exchange servers to Exchange 2007. Even though you don’t use Public Folders to store data in your environment, there’s one other reason why you might want to keep the Public Folder database mounted on your Exchange 2007 Server. As some of you may already know, Exchange 2007 no longer uses a Public Folder (or more specifically a System Folder named SCHEDULE+ FREE BUSY in your Public Folder hierarchy) to store free/busy information for the mailbox users in the organization. Instead free/busy information is stored directly in each user’s mailbox, and retrieved using a new web-based service called the Availability service. The advantage of this new approach is that there no longer are any 15 minute delays when free/busy time for a user is updated. Instead the update will happen instantly. So why would I want to keep the Public Folder database on my Exchange 2007 server, if free/busy information is retrieved using this new method? Well if you still have legacy Outlook clients (that is Outlook 2003 and earlier versions) running in your organization, these clients still need to use Public Folder method to retrieve free/busy information, since only Outlook 2007 supports the new Availability service.

If you don’t use Public Folders to store data and only have Outlook 2007 clients deployed in your organization, you can safely remove the Public Folder database, as you don’t have anything to use it for in that case. This also means you can skip the following steps.

Okay let’s get going with setting up a replica for the Public Folders on our Exchange 2003 Server that should be replicated with the new Exchange 2007 Public Folder database. In order to do so we must use either the Exchange 2003 System Manager or the Exchange Management Shell (EMS). For the purpose of this example we’ll use the Exchange 2003 System Manager.

Note
Managing Public Folders using the Exchange Management Console (EMC) is not possible in Exchange 2007 RTM, but will be integrated with Exchange 2007 Service Pack 1.

To add the Exchange 2007 Public Folder database to the replica list on the Exchange 2003 Server, open the Exchange 2003 System Manager, then expand Administrative Groups > First Administrative Group > Folders > Public Folders as shown in Figure 3.2.

                                Figure 3.2: Public Folders in the Exchange 2003 System Manager

Now open the property page of each public folder, then click the Replication tab and add the Exchange 2007 to the replica list as shown in Figure 3.3.

                                                  Figure 3.3: Public Folder Replication Tab

Note
Exchange 2003 Service Pack 2 introduced a new Public Folder Settings Wizard which makes it a breeze to add servers to replica lists. So if you have a lot of Public Folders in your Public Folder tree. If you have thousands of Public Folders, you might want to use the Public Folder replica scripts located in the Exchange Scripts folder (which can be found under C:\Program Files\Microsoft\Exchange Server).

Even though you still have legacy Outlook clients (Outlook 2003 and earlier) in your organization, you don’t need to set up a replica for the SCHEDULE+ FREE BUSY or the OFFLINE ADDRESS BOOK system folder. This will be done automatically when deploying an Exchange 2007 Server in a legacy Exchange organization.

When all Public Folders have been replicated to the Exchange 2007 Server, you should remove the old Exchange 2000 or 2003 Server(s) from the replica lists. When any Public folder data has been removed from the respective Public folder instances, you can dismount the old Public Folder stores (E2k3 SP2 won’t let you remove the Public Folder store until the data is gone and it won’t get removed while it’s dismounted). You should verify that your clients are still capable of seeing Public Folder data as well as free/busy information and accessing the offline address book before you delete it though. If this is not the case, I recommend you wait a little longer so that you’re sure the replication has occurred properly.

Important:
Outlook Web Access (OWA) 2007 doesn’t include a GUI for accessing Public Folders, so in order to access Public Folders using Internet Explorer you must open a separate browser window and type https://FQDN/public. It’s important you’re aware of this missing feature!

Pointing Internet Clients to the Client Access Server

Now would be a good time to point any Internet clients that are OWA, EAS and RPC over HTTP (now called Outlook AnyWhere) in your organization to the Client Access Server running on the Exchange 2007 Server. If you’re using a firewall such as ISA server (which you do, right?), this change is done at your ISA Server firewall. If you for some reason don’t use an ISA Server in your DMZ, but perhaps a Check Point Firewall 1 or a wannabe firewall such as a Cisco PIX, you should do the redirection there. If you don’t have a firewall you should make the change on the external DNS server hosting your Internet domain.

Note:
If your ISA Server is configured to pre-authenticate your OWA users, you must change the Authentication method for the OWA virtual directory under Server Configuration > Client Access in the EMC to basic authentication, since it’s configured to use forms-based authentication by default.

So will any users with a mailbox on my Exchange 2000 or 2003 Server still be able to use OWA, Exchange ActiveSync or Outlook AnyWhere (formerly known as RPC over HTTP) to access their mailbox? Yes this will work just fine since the Client Access Server is backward compatible and will redirect the clients to the respective legacy mailboxes on the Exchange 2000 or 2003 server.

Note:
When you perform the above changes, your users will no longer be able to access their mailbox using Outlook Mobile Access (OMA), as OMA has been discontinued in Exchange 2007.

Moving Legacy Mailboxes to Exchange 2007

Alright we have reached the part where we’re going to move our legacy mailboxes from Exchange 2000 or 2003 Server to Exchange 2007. Doing so is a straightforward process and can be done using either the Move Mailbox wizard in the Exchange Management Console (EMC) or the Move-Mailbox cmdlet in the Exchange Management Shell (EMS). For the purpose of this article series we’ll use the EMC. So if it’s not already open, launch the EMC, then expand the Recipient Configuration work center and click the Mailbox sub-node. Now highlight all the legacy mailboxes as shown in Figure 3.4, and then click the Move Mailbox task in the Action Pane.

                     Figure 3.4: Selecting Legacy Mailboxes in the Exchange Management Console

This will launch the Exchange 2007 Move Mailbox wizard, where you need to specify the destination server, storage group and mailbox database. Select the Exchange 2007 Server in the drop down box (Figure 3.5), and then click Next.

                      Figure 3.5: Specifying the Exchange 2007 Server as the Destination Server

Now specify how you want to manage any corrupted messages found in a mailbox (Figure 3.6), then click Next.

               Figure 3.6: Specifying how corrupted messages in mailboxes should be managed

On the Move Schedule screen shown in Figure 3.7, select Immediately (unless you want the mailboxes to be moved automatically at a later time) and click Next.

Figure 3.7: Move Mailbox Scheduling Options

Finally click Move in order to start moving the legacy mailboxes to the Exchange 2007 Server (Figure 3.8).

                                              Figure 3.8: Move Mailboxes Summary Page

As is the case with the Move Mailbox wizard in Exchange 2003, the Exchange 2007 Move Mailbox wizard can move 4 mailboxes at a time, and only one instance of the wizard can run on a server.

When all the mailboxes have been moved to the Exchange 2007 Server click Finish in order to exit the Move Mailbox wizard, and then check that mail flow to/from the Internet to the mailboxes on the Exchange 2007 works as expected.

If you will be running in a co-existence environment for a period of time, it’s important to understand that mailboxes stored on an Exchange 2007 server must not be managed using the Active Directory Users and Computers (ADUC) MMC snap-in, but instead must be managed using the Exchange Management Console (EMC) or the Exchange Management Shell (EMS). However Exchange 2003 mailboxes can still be managed using ADUC.

Note:
If you want to move the Mailboxes using the Exchange Management Shell (EMS), you do so using the Move-Mailbox cmdlet. Using the Move-Mailbox cmdlet gives you a set of advanced options, among which the most interesting one is the option of specifying the number of mailboxes to be moved at a time (as you read earlier the Move Mailbox wizard is limited to 4).

Redirecting Inbound Mail to the Exchange 2007 Server

When all legacy mailboxes have been moved to the Exchange 2007 Server, we can point SMTP traffic (port 25/TCP) directly to the Exchange 2007 Server, so that inbound messages are routed directly to this server. It’s recommended to deploy an Edge Transport Server in your perimeter network (aka DMZ), and let this server route inbound messages to the Exchange 2007 server on your internal network. Instructions on how to deploy an Edge Transport server is outside the scope of this article series, but I’ll cover that topic in another article in the near future. If you don’t want to deploy an Edge Transport server, you should bear in mind that you need to change the Permission Groups settings on the Default <server> receive connector under the Server Configuration work center node> Hub Transport sub-node in the EMC so Anonymous users are allowed to connect to the Exchange 2007 Server as shown in Figure 3.9, otherwise you won’t be able to receive e-mail messages from other SMTP servers on the Internet.

Figure 3.9: Permission Groups Settings on the Default Receive Connector

In addition you should make sure that any Send Connectors under Organization Configuration > Hub Transport > Send Connector tab are configured so that they can send outbound mail (either using a smart host or DNS MX) properly (Figure 3.10).

                                                     Figure 3.10: Send Connector Settings

When the necessary changes have been made, we can delete the routing group connector which was set up to establish mail flow between the Exchange 2003 and 2007 Routing Groups. In order to do so you should expand Administrative Groups > First Administrative Group > Routing Groups > Connectors and right-click on the respective Routing Group Connector then select Delete in the context menu.

Note:
Officialy the correct way of deleting the routing group connectors is to use the Remove-RoutingGroupConnector cmdlet, but since Exchange 2003 version blocking doesn’t block deletes, you can also use the Exchange 2003 System Manager as well.

Since the Routing Group connector won’t be deleted in both ends, you also need to delete it under the Exchange Administrative Group (FYDIBOHF23SPDLT) > Exchange Routing Group (DWBGZMFD01QNBJR) > Connectors.

Decommissioning Exchange Legacy Servers

The final step is to decommission the Exchange 2000 or 2003 Server and we can consider the transition done. The Exchange 2003 server should be removed using the Exchange 2003 Setup program, which can be launched via Add or Remove Programs.

But before you begin uninstalling the Exchange 2003 Server, we first need to assign the Recipient Update Service (RUS) to our Exchange 2007 Server. Not because RUS should be used (in fact Exchange 2007 no longer uses RUS), but because the Exchange 2003 Setup program won’t let us uninstall Exchange 2003, before RUS has been assigned to another server. In order to assign RUS to the Exchange 2007 Server, open the Exchange 2003 System Manager, then expand the Recipients node and select Recipient Update Services. Now open the property page both for Recipient Update Service (Enterprise Configuration) and Recipient Update Service (domain), then click the Browse button under the Exchange Server text box and specify the Exchange 2007 Server instead, then click OK twice and close the System Manager.

Note
It's important you don't delete the recipient policies in the Exchange 2003 System Manager, since Exchange 2007 uses them when provisioning users. 

            Figure 3.13: Assigning the Recipient Update Service to the Exchange 2007 Server

Note:
Microsoft will release an Exchange 2003 hotfix, which will prevent one from reassigning the RUS to an Exchange 2007 server some time in the future. The reason being, this really is an invalid setting that should be blocked. Instead the recommendation will be to use ADSIedit to remove the enterprise RUS object.

Now we can continue uninstalling the server, so select Microsoft Exchange then click the Change/Remove button.

The Exchange 2000 or 2003 wizard will appear, click Next then select Remove in the Action dropdown box as shown in Figure 3.14. Click Next.

Note
If your organization relies heavily on Public Folders, you might want to leave the Exchange System Management Tools intact, as you can use them to administer Public folders on your Exchange 2007 server. Remember Exchange 2007 doesn't have a UI for Public Folder Management.

                     Figure 3.14: Exchange 2003 Installation Wizard Component Selection Page

On the Installation Summary page click Next and wait for the Exchange 2003 uninstallation process to complete.

Note
If the Exchange 2000 Setup files aren’t located on an accessible drive, network share, you will be prompted to insert the Exchange 2003 CD media during the uninstallation process.

When the uninstallation process has completed click Finish to exit the Exchange 2003 Setup wizard .

 Alright we’re done!

Note
If the Exchange 2003 uninstallation for some reason fails, it may be necessary to remove the Exchange 2003 Server by deleting the Server object in the Exchange System Manager or even via ADSIEdit if this isn’t possible. But please don't delete the respective legacy (Exchange 2003) Administrative Group, as the user's legacyDNs still points there, even though their mailboxes are being moved in a native organization. 

Conclusion

Doing a transition from an Exchange 2000 or 2003 Server to an Exchange 2007 in the same Active Directory Forest is a straightforward process, and since Exchange 2007 co-exists just fine with legacy Exchange servers, you can do the transition at your own pace. Co-existence support is laudable as a transition process typically happens in several phases. This is especially true if you’re going to do a transition from multiple legacy Exchange Servers to multiple Exchange 2007 Servers.

I like the fact that the Exchange 2007 Setup wizard knows when Exchange 2007 is deployed in an existing legacy Exchange organization, and in this case prompts you to create a routing group connector so that mail flow is established between the legacy routing group and the Exchange 2007 routing group. It’s also nice to see that Exchange 2007 Setup, in this case, creates a Public Folder database and automatically adds the Exchange 2007 Server to the OFFLINE ADDRESS BOOK and SCHEDULE+ FREE BUSY system folders replica lists, so you only have to concentrate on replicating Public Folders.

Finally it’s a real pleasure working with the Exchange 2007 Move Mailbox wizard (or Move-Mailbox cmdlet) in order to move legacy mailboxes to an Exchange 2007 Mailbox Server, but I must admit, support for Public Folder management in the Exchange 2007 Management Console GUI is missed.

HOW TO: Configure Exchange 2007 Local Continuous Replication

Within Exchange 2007 Local Continuous Replication enables you to store a passive copy of the Exchange Database(s) on a second disk set. In case of failing of the primary disk set you easily can set the passive database copy of the database as active and therefore minimize downtime of Exchange.

Prerequisites:

• A second disk or better RAID Array to store the passive copy of the Exchange database. Do not use a second partition on the same disk set on which the active copy of the Exchange database resides.
• Every storage group within Exchange must only contain one database

To enable Local Continuous Replication the following steps must be configured.

Open the Exchange Management Console. Expand “Server Configuration” and choose “Mailbox”. Mark the storage group for which you want to enable LCR. From the action pane select “Enable Local Continuous Replication”. 

On the Introduction screen click Next

Select the path for the LCR system files and replication logs
(should be located on the second disk set)

If the storage group already contains a mailbox database you must also select a path for the passive copy of the database.

Click Enable to active LCR for that storage group

Verify the health status of LCR

To verify the status of your LCR configuration, you can use the following CMDlet:

Get-StorageGroupCopyStatus –Identiy ‘Servername\StorageGroupName’

The output of the CMDlet is shown in the following picture:

Restore Database from LCR Copy

Important: Before you can restore a database, the database must be offline (dismounted). Otherwise the restore process will fail.

To restore a database from a LCR Copy, use the following CMDlet:

Restore-StorageGroupCopy –Identity ‘Servername\StorageGroupName’

The output of the CMDlet ist shown in the following picture:

When the restore was successful performed you can mount the production database again. Also keep in mind that after a successful restore of the production database LCR is no longer configured for that storage group. You have to reconfigure LCR again.

 Exchange Server 2007 CCR on Windows Server 2008 Failover Cluster

First off, we need to understand that putting up a CCR cluster requires several steps, which can be combined into four categories.

1. Configure the Hardware
2. Install and Configure the Operating System
3. Install and Configure the Failover Cluster Feature
4. Install and Configure Exchange Server 2007 on the Cluster

Configuring the hardware really isn't difficult since we are talking about CCR. There is no need for a Storage Array Network (SAN) with all of the issues around creating, presenting, and securing Logical Unit Numbers (LUNs) for cluster storage. What we need to do here is just purchase our servers with two Network Interface Cards (NICs) and two internal disks.

Installing and configuring the operating system is also pretty straight forward. We need to use either the Enterprise or Datacenter version of Windows Server 2008 on each node. Once the OS is installed, each node needs to be joined to the domain.

One of the most important steps is to configure the operating system on each node with the proper role and features that are prerequisites for clustering and supporting Exchange Server 2007.

The prerequisites include:

• Web Server (IIS) and its Required Features
• Web Server (IIS) Role Services which includes:
• ISAPI Extensions
• Basic Authentication
• Windows Authentication
• IIS 6 Management Compatibility
• Windows Powershell

These prerequisites can be installed through the GUI or through the command line. For command line, run the following commands:

• ServerManagerCMD -i Web-Server
• ServerManagerCMD -i Web-ISAPI-Ext
• ServerManagerCMD -i Web-Metabase
• ServerManagerCMD -i Web-Lgcy-Mgmt-Console
• ServerManagerCMD -i Web-Basic-Auth
• ServerManagerCMD -i Web-Windows-Auth
• ServerManagerCMD -i PowerShell

The Web Server prerequisites are demonstrated in this IISPrerequisites recording, while the Windows Powershell install is shown in this Powershell recording using the GUI to install them.

The next step for the operating system configuration includes setting up the networks. The public network, also referred to as the client access point (CAP), is configured just like any other server. The network used for intracluster communications should be configured so that each NIC (one per node) uses a private IP address range and should not have a default gateway. It is a good practice to rename the networks so there is no confusion regarding their use.

Many cluster administrators will tune the intracluster communication network (also known as the private network or heartbeat network) so it is not configured with unnecessary services. For example, the private network should be configured as follows and as shown in this network clip:

• Clear the checkbox for Client for Microsoft Networks
• Clear the checkbox for QoS Packet Scheduler
• Clear the checkbox for File and Printer Sharing for Microsoft Networks
• Clear the checkboxes for the Link-Layer Topology options
• Clear the checkbox for Register this connection's address in DNS
• Clear the checkbox for Enable LMHOSTS Lookup
• Select the radio button for Disable NetBIOS over TCP/IP

Install and configure the failover cluster feature is the third major step in configuring our CCR cluster. This step is pretty easy. All we need to do here is add the Failover Cluster feature to each of our nodes so that they can be part of a cluster. The clip shows the steps of installing this feature. Actually, the feature is already installed in the clip, but it is easy to see from the clip how the feature would be installed on each node. You can also run the feature installation from the command line by running:

• ServerManagerCMD -i Failover-Clustering

Now that the feature is installed, we can take the next step and actually create our cluster. The Create Cluster link can be used in a couple of different locations to create the cluster and configure it as shown in this Failover Cluster clip.

Once we have created the cluster, we need to change the quorum type to support CCR. The recommended quorum type is Node Majority with File Share Witness.

Installing Exchange Server 2007 on the Cluster is the second to last step. In this step, we run the setup program from the Exchange Server 2007 installation media. During the installation, we will select the custom installation option and select Active Clustered Mailbox Role. The option to select either Cluster Continuous Replication or Single Copy Cluster is next.

The last step is to run the setup program from the Exchange Server 2007 installation media on the other node and select the Passive Clustered Mailbox Role. The steps are the same for the passive node as for the active node with the exception of selecting the passive installation option.

Exchange 2007 SCR Step by Step Implementation

SCR (Standby Continuous Replication)

As its name implies, SCR is designed for scenarios that use or enable the use of standby recovery servers. SCR extends the existing continuous replication features and enables new data availability scenarios for Exchange 2007 Mailbox servers. SCR uses the same log shipping and replay technology used by LCR and CCR to provide added deployment options and configurations by providing the administrator with the ability to create additional storage group copies. SCR can be used to replicate data from stand-alone Mailbox servers and from clustered mailbox servers.

Basic Requirement

1. Servers role like HUB/CAS/MAILBOX should be available into DR as it already have into production

SCR (Standby Continuous Replication)

As its name implies, SCR is designed for scenarios that use or enable the use of standby recovery servers. SCR extends the existing continuous replication features and enables new data availability scenarios for Exchange 2007 Mailbox servers. SCR uses the same log shipping and replay technology used by LCR and CCR to provide added deployment options and configurations by providing the administrator with the ability to create additional storage group copies. SCR can be used to replicate data from stand-alone Mailbox servers and from clustered mailbox servers.

Basic Requirement

1. Servers role like HUB/CAS/MAILBOX should be available into DR as it already have into production

2. the paths must be the same for both machines

– If source server is c:\Server1\Data and C:\Server1\Logs then these paths must be available on the target server.

3. There is a hard coded 50 log lag between the Source and Target

– by default there is a 24 hour replay time which is configurable.

4. There can be only 1 database per storage group

5. The target server must have Exchange mailbox role installed, if this is a cluster it will be install as a passive node.

6. The target server must be in the same Active Directory domain

7. Active Directory site should be different based on location

8. AD Site connector should be created between two sites

9. Make sure that outgoing mail relay needs to be configured into DR site

10 . N/W Bandwidth should be considered as per requirement 2 to 4 Mbps enough

11. maximum 4 server can be configured for replication

Step by Step commands

1. Enable-Storagegroupcopy sourceserver\Storage group -StandbyMachine TargetServer -ReplayLagTime 0:0:0

Run this command on the target server (DR server). You will not be able to folders created on DR servers across WAN link where database size is large

Note: It might be because of AD site replication duration you will not be able to expected information asap so run Replmon everytime to syncronize directory. Add domain servers into Replmon and right with syncronize server with different AD site (domain controller)

2. Get-storagegroupcopystatus sourceserver\SSG –StandbyMachine TargetMachine

Always run Get-storagegroupcopystatus command to confirm replication status. It will be failed or supspended so no need to worry

3. There are

You can perform seeding in Microsoft Exchange Server 2007 by using the following methods:

a) Automatic seeding : An automatic seed produces a copy of a storage group's database on the target. Automatic seeding requires that log1 be available on the source. Automatic seeding only occurs during the creation of a new server, creation of a new storage group and database, or on a database that has never been backed up.

b)Seeding using the Update-StorageGroupCopy cmdlet You can use the Update-StorageGroupCopy cmdlet in the Exchange Management Shell to seed a storage group copy.

c)Manually copying the offline database This process dismounts the database and copies the database file to the same location on the passive node. If you use this method, there will be an interruption in service because the procedure requires you to dismount the database.

Automatic seeding not required to run

Update-StorageGroupCopy. It will automatically create all data and log file

If its not happening second method always userfull

4. Suspend-StorageGroupCopy sourceserver\Storagegroup -StandbyMachine Target Sever

Click yes to suspend

5 . Update-StorageGroupCopy hub-cas\SSG -StandbyMachine DRE2K7

You will see progress of data copy after performing this command so wait for some time apprx 24 GB across 2 mbps link will take 3 to 4 days . if you are happy with this go for manually offline data copy which is not recommended because services will be stopped for that particular domain

After this you will be able to see the database folder into DR mailbox server

6. Resume-StorageGroupCopy hub-cas\SSG -StandbyMachine DRE2K7

After this you will be able to see the log folder into DR mailbox server

Run - Get-storagegroupcopystatus sourceserver\SSG –StandbyMachine TargetMachine to check the status

7. Create new storage group into DR site with database then delete the contains of the folder after dismounting

database. I

8. Dismount production storage group where SCR replication enabled.

9. Restore-Storagegroupcopy SourceServer\Storagegroup\database -StandbyMachine TargetServer

This will make you available Data to mount into DR server

10. Run eseutil /mh to check database status on the DR server

11. If you found Dirty Shutdown then run

Run soft recovery command : eseutil /r e02 (e02 - you will this details on the storage properties)

12. Again run eseutil /mh to check status and you wil find clean shutdown. In case able see error message run again following command

ESEUTIL /R E02 (Zero Zero or whatever the logs file leads with) /l D:\SCR\SG2\Logs /S D:\SCR\SG2\Data (Ex - logs and database location)

ESEUTIL /R E02 (Zero Zero or whatever the logs file leads with) /l D:\SCR\SG2\Logs /S D:\SCR\SG2\Data /a

13. Check again status using ESEUTIL /MH

14. Use move-storagegrouppath

15 . Use Move-DatabasePath

16. Set-Mailboxdatabase SCRTARGET\RECOVER\RECOVERDB -AllowFileRestore:$true

17. Then mount DR server storage group

18. Run -->

Get-Mailbox -Database hub-cas\SSG\MBX-SSG |where {$_.objectClass -NotMatch '(SystemAttendantMailbox |ExOleDBSystemMailbox)'}| Move-Mailbox -ConfigurationOnly -TargetDatabase DRE2K7\DRSSG\DRMBX -Confirm:$false

Now access mailbox and you will find its working with successfull DR using SCR

Microsoft Exchange Server Pipeline Tracing

Pipeline tracing is very useful tool for troubleshooting SMTP messages going through Exchange Server Hub Transport or Edge Transport role. Once pipeline tracing is enabled, you can view the log files under C:\Program Files\Microsoft\Exchange Server \TransportRoles \Logs \PipelineTracing.

Pipeline tracing only trace those messages which are sent from the SMTP address specified with the PipelineTracingSenderAddress parameter. It will not generate log files for other e-mail addresses. The SMTP address can be internal or external to your Exchange organization.

We can use <> parameter with PipelineTracingSenderAddress to generate log files for messages, like journal reports, automatic replies, NDRs and so on.

How to enable Pipeline Tracing

Syntax

Set-TransportServer <Identity> -PipelineTracingEnabled <$True | $False>

Example

Set-TransportServer Server1 -PipelineTracingEnabled $True

How to configure Pipeline Tracing Log Directory location

The default pipeline tracing log directory location is C:\Program Files\Microsoft\Exchange Server\TransportRoles\Logs\PipelineTracing. We can change this location by running below command:-

Syntax

Set-TransportServer <Identity> -PipelineTracingPath <LocalFilePath>

Example

Set-TransportServer <Identity> -PipelineTracingPath D:\Pipeline Tracing Logs

How to configure the Pipeline Tracing Sender SMTP address

Syntax

Set-TransportServer <Identity> -PipelineTracingSenderAddress <SMTPAddress>

Example

Set-TransportServer <Identity> -PipelineTracingSenderAddress sachink@techpeoples.net

How to configure Pipeline Tracing Sender Address to capture messages generated by e-mail servers

Syntax

Set-TransportServer <Identity> -PipelineTracingSenderAddress "<>"

Example

Set-TransportServer Exch01 -PipelineTracingSenderAddress "<>"

Troubleshooting Transport agents

In some cases, we have to ensure that the agent is doing its job within a message. We can accomplish this using the Pipeline tracing feature which allows us to create an exact snapshot of a whole message before and after it encounters each transport agent. Each step of the process is kept in a directory for troubleshooting reasons.

To verify the usefulness of Pipeline tracing, we will create a transport rule at Edge level that adds text into the subject field. After that we are will use Pipeline tracing to track the Sender xx@xx.com.br so we can validate Transport Agent processes in the messages sent from xx@xx.com.br.

To create Pipeline tracing, we can use the following cmdlet:

Set-TransportServer <server> -PipelineTracingEnabled:$true –PipelineTracingSenderAddress <smtp address or X500 address>

Where:

PipelineTracingEnabled: we are going to enable Pipeline tracing

PipelinetracingSenderAddress: for the users in the same site we can use either X500 address or SMTP and for external we use SMTP address.

In Figure 12 we can see the execution of the Set-TransportServer cmdlet. By default the directory where pipeline tracing generates the files is located under <Exchange Server directory installation>\TransportRoles\Logs\Pipeline Tracing\MessageSnapshots\<ID>\

                                  Figure 12: Enabling Pipeline tracing for the sender xx@xx.com.br

In the above mentioned directory we can see all of the snapshots of the single message that has pipeline tracing enabled (Figure 13). Each file has information about Transport agents.

                                         Figure 13: The directory where Pipeline tracing is saved

If we open any message within the directory using Notepad, we can see a header called X-Message-Snapshot-source that says which SMTP Event is being used and which Transport Agent is being applied on the message. We can see that the message sent from xx@xx.com.br for an internal user, received the Transport rule at Edge level when it was passing through the SMTP pipeline called OnEndofData in the Edge Transport Server, as shown in Figure 14.

Figure 14: The specific SMTP Event and the Transport Agent shown through pipeline tracing

After validation of the Transport Agent processes, you should turn off pipeline tracing, as shown in the Figure 15.

                    Figure 15: Disabling the Pipeline tracing using the Set-TransportServer cmdlet

Notes about pipeline tracing:

• Pipeline tracing copies all the message content. For security reasons you should define a directory protected by ACLs only for authorized personnel.
• Do not enable pipeline tracing for long periods of time, because there are a lot of verbose files that will be created and may cause disk space problems.
• To find the X500 address for internal users, you can use the Message Tracking log file.
• After disabling pipeline tracing, it is highly recommended that you delete all the information collected in that period.

Logging SMTP Protocol Activity in Exchange 2010 and Exchange 2007

Exchange Server 2007 has its own SMTP stack, and what I like to think of as smarter or more intelligent Receive Connectors (these are protocol listeners, roughly equivalent or comparable to the SMTP Virtual Server we’ve known from Exchange Server 2003/2000 – Bharat).

Not enabled by default

I hoped to see (SMTP) protocol logging turned on by default on these connectors, but this is one aspect that hasn’t changed. Yes, SMTP logging is still not enabled by default! You have to remember to enable SMTP logging on transport servers.

Enable protocol logging on a Receive Connector

To enable protocol logging on Receive Connectors, use the following command:

Set-ReceiveConnector “Connector Name” -ProtocolLoggingLevel verbose

If you’re wondering whether there are any choices for the logging level – there aren’t. It’s either verbose or none.

To enable protocol logging from the Exchange console (EMC):

1. Expand the Server Configuration | Hub Transport node
2. Select the Hub Transport server you want to configure, and then select the Receive Connector -> Properties
3. On the General tab, change the Protcol logging level to Verbose, as shown in the screenshot below.

                                       Figure 1: Enabling SMTP logging on a Receive Connector

Enable protocol logging on a Send Connector

Unlike Exchange Server 2003/2000, you have to enable logging separately for Send Connectors (used to send mail outside the Exchange organization, Send Connectors are equivalent of SMTP Connectors in Exchange 2003/2000), using the following command:

Set-SendConnector “Send Connector Name” -ProtocolLoggingLevel verbose

To enable protocol logging on a Send Connector using the EMC:

1. Expand the Organization Configuration | Hub Transport node
2. On the Send Connectors tab, select the Send Connector -> properties
3. On the General tab, change the Protocol logging level to verbose, as shown in the following screenshot.

                                         Figure 2: Enabling SMTP logging on a Send Connector

Besides the visible Receive and Send connectors, an invisible Send Connector lurks under the hood – used to transport messages within the organization, between Hub Transport servers, Edge Transport servers, and Exchange Server 2003/2000 servers. It’s the Intra-Organization Send Connector. You won’t see it in the console, or in the shell if you use the get-SendConnector command. To configure protocol logging for this Intra-Organization Send Connector:

Set-TransportServer “TRANSPORT SERVER NAME” -IntraOrgConnectorProtocolLoggingLevel verbose

Where do protocol logs reside?

Unlike Exchange Server 2003/2000, which maintain separate protocol logs for each SMTP Virtual Server, all Receive Connectors share SMTP receive logs. Similarly, Send Connectors share SMTP send logs.

Receive Connector logs are located in:

• Exchange 2010: \Exchange Server\V14\TransportRoles\Logs\ProtocolLog\SmtpReceive
• Exchange 2007: \Exchange Server\TransportRoles\Logs\ProtocolLog\SmtpReceive

Send Connector logs are located in:

• Exchange 2010: \Exchange Server\V14\TransportRoles\Logs\ProtocolLog\SmtpSend
• Exchange 2007: \Exchange Server\TransportRoles\Logs\ProtocolLog\SmtpSend

Change SMTP log paths

It’s generally a good idea to not locate Exchange data, including Exchange mailbox and public folder databases, transaction logs, and all other logs on the system drive.

To change the path of SmtpReceive logs:

Set-TransportServer “TRANSPORT SERVER NAME” -ReceiveProtocolLogPath “C:\New SmtpReceive Log File Directory”

To change the path of SmtpSend logs:

Set-TransportServer “TRANSPORT SERVER NAME” -SendProtocolLogPath “C:\New SmtpSend Log File Directory”

Permissions on the log directory

If you do decide to change the path, ensure the new directories/folders exist with appropriate permissions:

• Administrator: Full Control
• System: Full Control
• Network Service: Read, Write, and Delete Subfolders and Files

Controlling protocol log size

Since SMTP support was provided by IIS, SMTP logging options were also controlled in IIS. IIS did not provide a way to control the disk space occupied by SMTP logs. As a a result, you had to archive or clean up the log directory manually, or automate it using a script. SMTP logging was one of the culprits that contributed to disk volumes on Exchange 2003/2000 servers running out of disk space.

In Exchange 2010/2007, you can control the following options:

• ReceiveProtocolLogMaxAge: the maximum age of a receive log. Files older than the specified age are removed.
• ReceiveProtocolLogMaxDirectorySize: the maximum size of the directory. This ensures the protocol logs for a Receive Connector do not exceed a fixed size.
• ReceiveProtocolLogMaxFileSize: the maximum size of a single log file. When the active log file reaches this size, it’s rolled over and a new log file created.

Similarly, protocol logs for Send Connectors can be controlled using the following parameters:

• SendProtocolLogMaxAge
• SendProtocolLogMaxDirectorySize
• SendProtocolLogMaxFileSize

The default parameters work for most deployments, and ensures you don’t have to worry about purging the logs manually over time, or scheduling a script to do this periodically. You may want to observe the logs created in your environment to determine if these are sufficient, depending on the traffic and number of days worth of logs you want to keep around for troubleshooting.

SMTP logs are an important troubleshooting tool – enabling SMTP logging after the fact isn’t any help when troubleshooting SMTP mail flow.

Exchange Server 2007: Design and Deploy Disaster Recovery Settings - Use Dial-Tone Restores

Problem: If your server goes down and you need to restore email functionality without the concern of previous mailbox data, is there some way to restore email to your users without worrying about the mailbox data until a later time? How would that be performed?

Solution: Because the mailbox itself is part of Active Directory and the actual mailbox data is within the Exchange databases, it is possible to restore an Exchange server quickly to the point of users having the capability to send and receive email again. This provides time for the actual database restoration to occur. This is called a dial-tone restore (also called the recover now, restore later method).

Some of the stress of this method comes from merging the two sets of data (the old recovered data with the new email sent back and forth in the dial-tone recovery database).

Keep in mind that there are several reasons to perform a dial-tone restore. One is that you have a failed server, and you need to use an alternate server for the restoration of email services. Another is that you simply have a failed or corrupt database that is just going to take too long to restore and your users need email now (so you might use the same server). You have to make some decisions regarding leaving the data on the recovery server or moving it back when the crisis is under control.

Perform a Dial-Tone Recovery on the Same Server

To mimic the failed database, you can dismount the database and locate the database file and delete it (or move it to another location). In Step 2, you create a new database, which does not require mailbox movements because this is the same server and storage group you are recreating. This gives you the time to recover the data to the RSG while your users have up-and-running mailboxes.

To achieve dial-tone recovery, perform the following:

1.	If you have any log files from the failed database, you might want to move them to another location. They might be useful.
2.	Open the EMC.
3.	Attempt to mount the database and you receive the message shown in Figure 1 regarding the creation of an empty database. Click Yes to continue. Figure 1. Warning message of an empty database.
4.			From the Navigation Tree, expand the Toolbox work center.
5.		From the options, open the Database Recovery Management tool.
6.			Create a recovery group for the failed database.
7.			Restore the backup you have of the failed database, but do not mount the database if you have any log files from the failed database. Place those in the proper location before mounting.
8.			Mount the database from the task center.
9.			From the task center, select the Swap Databases for Dial-Tone scenario.
10.				Make sure the database is correct.
11.				Select Gather Swap Information.
12.				Select Perform Swap Action after confirming the information is accurate.
13.				After the procedure is complete, the configurations for the original and recovery databases are swapped and remounted.

Swapping databases provides users the original mailboxes they had, but moves over the dial-tone database. The two should be merged to ensure no data is lost.

To do this, go back to the task center and use the Merge or Copy Mailbox Contents settings (just as you would with a normal recovery group) and merge that content over to the recovered and swapped database.

When you are done, remove the RSG.

Perform a Dial-Tone Recovery on a Different Server

The steps for a “new server” dial-tone recovery are almost identical to that of a “same server” dial-tone recovery. Here are some changes and additions:

·         You have to create a new storage group and database with the names of the original on the new server.

·         You need to use the Move-Mailbox -ConfigurationOnly command to point the user’s mailboxes to the new database. In doing this, you are moving the configuration information, but the data will be restored with the restoration process. The official command is:

Get-Mailbox -Database <Mailbox Server Name\Storage Group

Name\database name> | Move-Mailbox -ConfigurationOnly -TargetDatabase <Mailbox Server

Name\Storage group

Name\Dial Tone database

name>

                     

·         If you plan to restore your backup to the new server, the steps are the same as previously explained. If you plan to restore your backup to the failed server, you need to use the same EMS command to move the mailboxes back when you are done.

Note

Users might be somewhat surprised when they have no email. However, they might be even more surprised if they have email but no access to all their messages. You will want to alert them with a message about what is happening. Some people receive a message that gives them the capability to see their old data if they have Outlook 2007 in cached mode. However, if they choose to use old data, they cannot send and receive. They will be presented with the option to use a temporary mailbox, and this is the option they need to select to send and receive messages.

Relieving Exchange 2007 Transport Back Pressure

Server applications like MS Exchange can be very resource intensive. Resource consumption can be anticipated to a certain extent. For example given the maximum mailbox size one could calculate the necessary HDD space required by the store. The necessary processing power and RAM could also be computed from vendor data and experience.

However even if a server is commissioned by the book, guaranteeing the machine is always able to handle its load is nearly impossible. A server functioning perfectly could run into an extraordinary load peak that puts under test its true limits. This is what makes server monitoring so important.

Exchange 2007 built new monitoring capabilities right into the product core. Its new transport now includes a real-time monitoring feature referred to as Back Pressure and is implemented by both Hub and Edge Transport server roles. This monitors the key resources the transport is dependent on. If a resource shortage is detected, Back Pressure kicks in reducing the amount of load handled. This equates in stopping the delivery of some messages in an attempt to retain minimal functionality avoiding further resource depletion. Administration can then correct the situation without loosing the transport altogether. Of course if no action is taken, full Back Pressure could take effect blocking message flow completely.

A Close Encounter with Back Pressure

It did not take me long to discover Back Pressure. My virtual machines are often well below the recommended requirements, especially if installed for taking article screenshots.

When Back Pressure goes active the first thing you will notice are stuck messages. Since Exchange is based on the SMTP transport, we can see how Back Pressure rejects SMTP connections. Telneting Exchange on port 25 we got the rejection response:
450 4.3.1 Insufficient System Resources

The event IDs 15001, 15002 and 15003 logged by MSExchangeTransport are more informative. Here is an example:

Event Type: Warning

Event Source: MSExchangeTransport

Event Category: ResourceManager 

Event ID: 15002

Date: 17/07/2007

Time: 13:53:35

User: N/A

Computer: BANG

Description:

The resource pressure is constant at High. Statistics: 

Queue database and disk space ("C:\Program Files\Microsoft\

Exchange Server\TransportRoles\data\Queue\mail.que") = 68% 

[High] [Normal=45% MediumHigh=47% High=49%]

Queue database logging disk space ("C:\Program Files\Microsoft\

Exchange Server\TransportRoles\data\Queue\") = 68% [Normal] 

[Normal=89% MediumHigh=91% High=93%]

Version buckets = 1 [Normal] [Normal=40 MediumHigh=60 High=100]

Private bytes = 14% [Normal] [Normal=71% MediumHigh=73% 

High=75%]

Physical memory load = 50% [limit is 94% to start dehydrating 

messages.]

Inbound mail submission from other Hub Transport servers, the Internet, the Pickup directory, the Replay directory, and the Mailbox server, if it is on a Hub Transport server, has stopped.Loading of e-mail from the queuing database, if available, continues.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp

This event shows the consumption level of the 5 resource vectors monitored. Here is the description for each of these taken from TechNet (see links below):

1. Free space on the hard disk drive that stores the message queue database.
2. Free space on the hard disk drive that stores the message queue database transaction logs.
3. The number of uncommitted message queue database transactions that exist in memory.
4. The memory that is used by the EdgeTransport.exe process.
5. The memory that is used by all processes.

Consumption is reported in terms of percentages and tested against the configured limits. In all 3 limits are considered, Normal, Medium and High. Back Pressure will gradually start to reduce resource consumption as these limits are exceeded. For an explanation on how the default limits are computed and on the various Back Pressure stages check the references section.

Relieving the Pressure

Hub and Edge Transport server read their Back Pressure and various other settings from EdgeTransport.exe.config file. This is an XML file located under the Exchange bin directory. From here we could change the Back Pressure limits and even disable it altogether. However this is not wise. Back Pressure is a good feature, and we should make sure the necessary resources are available.

The file is composed of a series of name value pairs. Each setting is held within an 'Add' XML element. Some of the Back Pressure settings follow. However note that this file contains many other settings not related to this feature.

EnableResourceMonitoring - Turn on/off Back Pressure

ResourceMonitoringInterval - Amount of time between subsequent resource level tests

PercentageDatabaseDiskSpaceUsedNormalThreshold, PercentageDatabaseDiskSpaceUsedMediumThreshold, PercentageDatabaseDiskSpaceUsedHighThreshold - Normal, Medium and High thresholds for transport message queue disk space

For a full list of these settings see the references. Here our goal is not that of tweaking the monitoring settings, but is rather that of providing the necessary resources.

As a minimum we should make sure the transport has a good amount of disk space, memory and processing power. Keep in mind the machine resources are being shared between all applications running on the machine. The machine may be running multiple Exchange roles. Third party message hygiene applications will also find the transport to be a good place where to plug. All of these will add-up rendering the resource requirements harder to predict without proper testing.

Thanks to the 64-bit address space we should have plenty of memory extensibility. Exchange will by default put all databases on the installation drive. So hard disk space could be more of an issue. Luckily relocating the transport message queue database and transaction logs are an XML element away.

1.      Create the directory where the new transport database/logs are to be held. Exchange won't create the directory tree for you.

2.      Assign Full Control permissions to the Network Service, System and Administrators Group over the new directory location.

3.      Open EdgeTransport.exe.config located under the Exchange bin directory.

4.      Edit the XML elements for QueueDatabasePath and QueueDatabaseLoggingPath. The former identifies the queue database path and the latter identifies that for the transaction logs.

By default both point to:
<Exchange dir>\ TransportRoles\data\Queue

We will now change these to the new disk location, for example:
<add key="QueueDatabasePath" value="D:\Exchange\Queue" />
<add key="QueueDatabaseLoggingPath" value="D:\Exchange\Queue" />

Be careful when editing XML files. It is best to use an appropriate XML editor that ensures the XML is well formed.

5.      Restart the Microsoft Exchange Transport service.

Exchange will now create new queue and transaction logs. The old files won't be deleted. After verifying that the new files were indeed created, then old files can be deleted manually. Here is the list of files involved:

Message Queue:
Mail.que and Trn.chk

Transaction Logs:
Trn.log, Trntmp.log, Trnnnn.log, Trnres00001.jrs, Trnres00002.jrs, and Temp.edb

Alternatively we could move the files to the new location. In this case instead of restarting the Microsoft Exchange Transport service in one step, we would first stop, move the files and then start the service again.

Final Tips

Back Pressure adds some native resource monitoring to the Exchange Hub and Edge transports. This is "must have" functionality for such a business critical application. Tweaking the Back Pressure settings is possible but not recommended. Instead we should make sure to provide the necessary resources.

If the machine keeps hitting the limits it is likely we have an overloaded server. Exchange allows for running the transport role on multiple boxes. In this manner the load can be split bringing resource consumption to normal levels again.

Lost Log Resilience and Transaction Log Activity in Exchange 2007

This topic discusses lost log resilience (LLR) and a companion function called log roll. These features were introduced with the release to manufacturing (RTM) version of Microsoft Exchange Server 2007. The behavior of these features has been modified in Exchange 2007 Service Pack 1 (SP1). These features are present on all mailbox servers. However, the behavior of these features depends on the configuration of the mailbox server and the version of Exchange 2007 that is installed.

 Lost Log Resiliency

In Exchange 2007, an internal component of Extensible Storage Engine (ESE) called LLR enables you to recover Exchange databases even if one or more of the most recently generated transaction log files have been lost or damaged. By default, LLR is enabled on all Exchange 2007 mailbox servers. LLR enables a mailbox database to mount even when recently generated log files are unavailable. One cause of unavailable log files is a lossy failover in a cluster continuous replication (CCR) environment, which is also known as an unscheduled outage. 

Note:

In a continuous replication environment, LLR is enabled only for the active copy of a database. LLR is not used by the passive copy because passive databases are always kept as up-to-date as possible.

The order of write operations of Exchange data is always memory, log file, and then database file. LLR works by delaying writes to the database until the specified number of log generations have been created. LLR delays recent updates to the database file for a short time. The length of time that writes are delayed depends on how quickly logs are being generated.

In the event of a failover, the passive copy of the databases can be automatically mounted by the Microsoft Exchange Information Store service if the number of lost logs is fewer than the allowable amount as configured by an administrator. An administrator determines the maximum number of logs that can be lost before the database cannot be mounted on a failover by setting the AutoDatabaseMountDial parameter. This parameter, which is represented in the Active Directory directory service by an Exchange attribute called msExchDataLossForAutoDatabaseMount, has three values: Lossless, Good Availability, and Best Availability. Lossless is 0 logs lost, Good Availability is 3 logs lost, and Best Availability, which is the default, is 6 logs lost.  When you configure the system for Good Availability or Best Availability, do not use spaces (for example, use GoodAvailability and BestAvailability).

 Transaction Log Roll

A mechanism called log roll is used to further minimize data loss. Log roll works by periodically closing the current transaction log file and creating the next generation. This mechanism helps LLR, and in turn CCR, to reduce data loss that results from lost log files, primarily after a lossy failover.

Important:

The log roll mechanism does not generate transaction logs in the absence of user or other database activity. In fact, log roll is designed to occur only when there is a partially filled log.

Rolling a log forward means that the current (Exx.log) log file is closed and a new transaction log file is generated, even if the current log file is not full.

The log roll behavior is based on the value of the LLR depth. In a CCR environment running Exchange 2007 RTM, the LLR depth is a numeric value equal to 1 plus the tolerable number of lost logs, as specified by the value of the AutoDatabaseMountDial parameter. For example, if the value of the AutoDatabaseMountDial parameter is 6, indicating the system is configured for Best Availability, the value of the LLR depth is 7.

In a CCR environment running Exchange 2007 SP1, the LLR depth is hard-coded with a value of 10, regardless of the value of the AutoDatabaseMountDial parameter.

In both Exchange 2007 RTM and SP1, the LLR depth is hard-coded with a value of 1 for all mailbox servers that are not in CCR environments (for example, stand-alone mailbox servers with or without LCR and single copy clusters).

Log roll will occur after the system has been idle for a calculated period of time. To calculate when log roll should occur, the system uses the following formula:

[15 (minutes) ÷ LLR Depth value] = Frequency of log roll activity (in minutes)

You can then divide 1,440 (the number of minutes in each day) by the frequency of log roll activity to determine the maximum number of log files per storage group that would be generated each day as a result of log roll activity.

For example, in CCR environments running Exchange 2007 SP1, the LLR depth is 10. Thus, log roll activity occurs every 1.5 minutes, and the maximum number of log files generated per storage group each day as a result of log roll activity is 960.

 Log Roll Size

For a log roll of significant size to develop in a storage group, the following conditions must be met:

• The storage group must have a mailbox database.
• The storage group must have little user activity that creates transaction logs.
• The storage group must have one or more mailboxes that are frequently logged on to by a process or by an application.

The maximum number of log files that will be generated each day for an idle storage group depends on the configuration of the mailbox server. The maximum number of log files per idle storage group for each mailbox server configuration is listed in the following table.

Maximum number of log files per idle storage group for each Exchange 2007 RTM mailbox server configuration

Mailbox server configuration	Maximum number of transaction log files generated per day by an idle storage group
Stand-alone (with and without LCR) Single copy cluster CCR with Lossless availability	96
CCR with Good Availability	384
CCR with Best Availability	672

Maximum number of log files per idle storage group for each Exchange 2007 SP1 mailbox server configuration

Mailbox server configuration	Maximum number of transaction log files generated per day by an idle storage group
Stand-alone (with and without LCR) Single copy cluster	96
CCR with Lossless, Good, and Best Availability	960

Mailbox servers generally create more transaction logs than the value shown in the preceding tables because of user activity, online maintenance, and other factors.