Hey all,

We use and support a backup product by StorageCraft called ShadowProtect. A huge feature with this product is the HIR or Hardware Independent Restore which allows a backup image to be restored to any hardware - Very useful when your server melts down and you are not able to get the same mainboard/disk drives etc etc that ultimately would cause the OS to crash.

Long story short; I was restoring a Domain Controller using this process and essentially checking that the restored Domain Controller is functional. In the event a restore was required, no problems would be encountered. In the case, big problems were encountered as the Domain Controller that I restored was part of a multi DC environment with the various DC roles split amongst the servers - thus without all DC's up and running, AD (Active Directory) does not function.

This led me down the path of consolidating these non present Domain Controllers onto a single DC that has been restored from backup and so I designed a case scenario and procedure which I thought I would offer to you guys for use/comment.

As with all disaster scenario's - we really hope this never happens!

----------------------------------------------------------------------

Disaster Recovery: Consolidate a multiple Domain Controller environment on to a single Domain Controller.
Scenario:
A client’s server room was consumed by fire last night, destroying all server computers, including all 3 Domain Controllers (DC-1, DC-2, and DC-3). Server FSMO roles were divided among each of the Domain Controllers, requiring all 3 DC’s to be operating for full AD functionality, and network data/shares.
DC-2 & DC-3 are contained within different ‘sites’ in AD.
Workstations and other auxiliary equipment are intact.
Objective:
Restore network functionality, including network data/shares and AD, using only the backups available and new server hardware (single box).
Consolidate DC roles from DC-2 & DC-3 onto the restored server (DC-1) and test AD/domain functionality.
Considerations:
Only a single ‘full system backup’ of DC-1 is available offsite from the day before.
A complete backup of all network data is available also, independent of server & system state.
DC-1 is the DNS & DHCP server.
Network data is shared as ‘data’

DC-1 is the restored server, on which everything must be consolidated. Replace “DC-1” with the relevant server name.
DC-2 & DC-3 are the destroyed servers that need to be removed from the domain. Replace these names with the relevant server name.
Process:
Restore Server Image & HIR Process
1) Restore DC-1 from the available backup as per the ShadowProtect HIR Instructions, on to the new server hardware.
2) (As part of the HIR Instructions) on the first boot, press F8 to invoke the Windows Boot Menu and select ‘Directory Services Restore Mode’. If Windows boots into normal mode on the first boot, you will need to restore the system volume from backup and start the process over.
3) Login to the console using DSRM credentials.
4) Go to the advanced tab on System Properties and select ‘Environment Variables’.
5) Select ‘New…’ on the System Variables Section.
6) Set the Variable Name as “DEVMGR_SHOW_NONPRESENT_DEVICES” (case sensitive)
7) Set the Variable Value as “1” & Apply the changes
8) Go to Device Manager, click on the View menu and select ‘Show Hidden Devices’
9) Remove or uninstall all hidden/non-present devices (greyed out icon), most importantly the NIC’s.
10) Once non-present devices are removed, return to the System Variables and remove the “DEVMGR_...” variable you have just created and apply the changes.
11) Open Network Connections and view the properties of the NIC.
12) Configure the NIC to replicate the settings configured on the production server.
13) Double check to ensure the Restore server NIC & the original server NIC are identical.
14) Reboot the server into normal mode.
Repair Active Directory & Remove Offline Servers
15) Log in to the server using domain credentials, at this stage there may be system errors and problems with AD as it is reliant on DC-2 & DC-3 which are no longer available – Ignore all errors/problems for the moment.
16) Open AD Users & Computers
17) For both DC-2 & DC-3 objects, right click and select delete.
18) For both DC-2 & DC-3 select the option “DC is permanently offline and can no longer be demoted”.
19) Start a command prompt and type: ntdsutil
20) At the “ntdsutil:” prompt type: metadata cleanup
21) At the “metadata cleanup:” prompt type: connections
22) At the “connections:” prompt type: connect to server DC-1.domain (the name of the restored server in FQDN format)
23) Type: q
24) At the “metadata cleanup:” prompt type: sel op tar
25) At the “select operation target:” prompt type: list domain
26) A list of available domains will appear with a corresponding number
27) At the “select operation target:” prompt type: sel dom 0 (the number corresponding to the domain we are consolidating)
28) At the “select operation target:” prompt type: list sites
29) A list of available sites will appear with a corresponding number
30) At the “select operation target:” prompt type: sel sit 0 (the first site)
31) At the “select operation target:” prompt type: list servers in site
32) A list of available servers within the selected site will appear with a corresponding number
33) At the “select operation target:” prompt type: sel ser 0 (the first server that has been destroyed, in this scenario DC-2)
34) At the “select operation target:” prompt type: q
35) At the “metadata cleanup:” prompt type: remove selected server
36) Confirm this action on the GUI dialog box that will appear.
37) At the “metadata cleanup:” prompt type: sel op tar
38) Repeat steps 31 – 35 for each server within the selected site that has been destroyed, in this scenario DC-3 is the next server to be removed.
39) At the “metadata cleanup:” prompt type: sel op tar
40) Repeat steps 30 – 38 for each site within the domain, removing destroyed servers from each site as applicable.
41) At the “metadata cleanup:” prompt type: q
42) At the “ntdsutil:” prompt type: roles
43) At the “fsmo maintence:” prompt type: connections
44) At the “server connections:” prompt type: connect to server DC-1 (the current server)
45) At the “server connections:” prompt type: q
46) At the “fsmo maintence:” prompt type: transfer dom nam mas
47) Confirm this action on the GUI dialog box that will appear.
48) At the “fsmo maintence:” prompt type: transfer inf mas
49) Confirm this action on the GUI dialog box that will appear.
50) At the “fsmo maintence:” prompt type: transfer pdc
51) Confirm this action on the GUI dialog box that will appear.
52) At the “fsmo maintence:” prompt type: transfer rid mas
53) Confirm this action on the GUI dialog box that will appear.
54) At the “fsmo maintence:” prompt type: transfer schema mas
55) Confirm this action on the GUI dialog box that will appear.
56) At the “fsmo maintence:” prompt type: q
57) At the “ntdsutil:” prompt type: q
58) At the command prompt type: exit
59) Reboot the server
Checking Server & AD Functionality
60) Complete the Domain Controller Checklist to ensure Domain Functionality using DC HIR Restore Checklist I created.
61) Once the Domain Controller Checklist has been completed successfully, AD is now fully functional and consolidated to a single server.
Final Configuration of Network Data and Offline Server Roles
62) Restore Network Data to DC-1 and configure shares as required. As no backup was available of the file server, shares and permissions will need to be recreated.
63) Deleted servers will still be shown in ‘Sites & Services’ and should be deleted with the right-click menu. As no backup of DC-2 & DC-3 is available, any site configuration will need to be recreated when a new DC is promoted.
At this point, Active Directory has been fully restored and consolidated on to the only Domain Controller now in production. The network data has been restored from backup and the shares have been reconfigured.