ActiveRoles Server Management History Migration Wizard – A Personal Experience

We’re off to see the wizard – the Management History Migration Wizard, that is. As discussed in this previous post, we did not migrate our management history from ARS 6.5 at the same time we started having our admins start managing user accounts with ARS 6.7, and that caused no small amount of annoyance.

Today (Friday) is the day. The test accounts have been created, deprovisioned and in some cases, undeprovisioned. The SQL Server DBA has handed me scripts to back up the ARS databases. I have created Access Template Links so that the admins whose normal write privileges I am suspending this weekend can still unlock accounts and reset passwords. Access Template Links for normal privileges have been disabled. Admins have gotten spammed. Self Service users have seen the bulletin. IIS (w3svc) has been stopped on all the ARS servers.

It is time to do the Management History migration that did not happen earlier, as discussed in a previous post, ……

Quest’s own Knowledge Base gives the vague statement that this could take “in excess of 25 hours”, and I have not found anything more concrete. Therefore, I have allocated 60 hours for ARS to be unavailable.

Our stats:

ARS 6.5 DB: 94 GB, 3.75 million records to import
ARS 6.7 DB: 6 GB
SQL Server 2005, running on Windows Server 2003 (both DBs are in the same instance)

The local backup of the 6.7 DB took less than two minutes; the enormous 6.5 DB took about 20 minutes.

18:32 – And we’re off!

At 21:30, 55% of the first step was finished. At that pace, that part should be over about 00:30.

Saturday:

08:00 – it’s gotten through steps 1-3, and is now 23% through the actual data transfer.
08:05 – 887000
08:10 – 897500 – 10500 record per 5 minutes – 126000 per hour

estimated finish time for data transfer: Sunday, 6:00. Might as well go skiing at this lovely little place in the Bavarian Forest while the Wizard works its magic.

19:00 – Back from skiing; the snow was pretty good, and the Wizard is at 61%. A bit slower than hoped, but still on track for a Sunday morning finish.

Sunday:

10:20 – 81%. Oh, dear – this won’t be done by start of business Monday! My test accounts had their depro history available, but did not offer undo depro. What a disaster. Hmmm…. synthetic undepro using the xml history?
16:00 – 93% (whew), 250000 records to go – might be done around 19:00.
19:37 – DONE, 49 hours after I kicked off the Wizard, and 50 hours after I restricted everyone’s access to ARS and took SQL Server backups. This is exactly twice as long as the “sometimes in excess of 25 hours”, which is the only estimate I could find. So here is a real datapoint: it took 50 hours to import 2 years’ worth of management history for an ARS installation with about 50,000 enabled user accounts and 3.75 million changes.

New ARServer67 DB size: 80 GB. The ARServer65 DB was unchanged.
Just about everything looks to be in order – the deprovision information itself (edsvaDeprovisionReportXML) was brought over and looks right for my test objects, as does the surrounding Change History.

There is still that one fly in the ointment: the edsvaDeprovisionStatus attribute was not changed to 1, and Undo Deprovision is not available in the context menu; Deprovision is. This is disappointing, as this was the main goal of the import (aside from being able to dump the old database and 6.5 server). I’m seriously considering a sort of scripted Undeprovisioning, available for disabled objects with existing edsvaDeprovisionReportXML data.

But before I get into all that, I’ll restart the arssvc on both the 6.7 servers and hope for the best.

21:20 – Doh! No difference.

Monday:
6:00 – Enabled the Access Template Links I disabled Friday at 18:00, restarted w3svc so that the ARS web interface would be available to admins and data owners. Did NOT do the same on ARS 6.5 🙂

So far, no one has reported any funny business, so I’ll count the whole exercise as mostly harmless, even slightly helpful. At least I have the data from 6.5 in the same place as the current stuff.

So now what?

edsvaDeprovisionReportXML contains the changes to the following attributes:

description
displayname
edsaAccountIsDisabled
msExchHideFromAddressLists
altRecipient
showInAddressBook
edsaMailboxSecurityDescriptor

but not

authOrig
manager
homeDirectory
accountExpires

These last four are under the username of the ARS service account; the depro itself takes place under the username of the admin who initiated the depro request.

The four attributes not included in edsvaDeprovisionReportXML are changed a few seconds before the ones in the the Depro Report; they could be picked up by Get-QARSOperation -ChangedBefore [5 minutes after depro request] -changedAfter [5 minutes before depro request]

Could change depro policy to keep the attributes the same if “Deprovisioned” is in the homeDirectory or the DisplayName, then change undepro to truncate the depro+date info.

Any ideas would be GREATLY appreciated!

Link to my discussion on the Quest Communities ActiveRoles Server board: http://communities.quest.com/thread/21170

ActiveRoles Migration: Deprovisioning/Undoing Deprovisions, or, The Article I Wish I’d Read Before We Started Our Migration

If you read nothing else before you start an ActiveRoles Server upgrade, read Matt Hitchcock’s post about migrating deprovision information: http://sgitpro.com/2011/07/10/migrating-deprovision-information-with-activeroles-server/

Matt discusses an ActiveRoles Server 6.1 to 6.5 migration, but it would apply equally to 6.5 to 6.7 (our situation). This applies to migration scenarios where you are setting up a new database for the new version of ARS. If you are doing an in-place upgrade, it does not. As Matt says in his post, PLEASE try this in your test environment before doing it in production.

Because neither I nor my colleagues had read this, we are stuck keeping a version 6.5 server around for the foreseeable future to allow deprovision/undo deprovision operations on accounts that got caught in the gap between when the first 6.7 server was put in place and when we started having the help desk and local administrators use it.

Our Management History was never imported; it was something our consultant probably intended to do at a later date or to advise us to do, but it never happened, and I’m quite hesitant to do it now that 6.7 has been used for deprovisioning/undoing deprovisions. Here’s what I’ve told the help desk and administrators around the world:

  • If “Undo Deprovision” is not available for a deprovisioned account, that means it was done in 6.5 after 6.7 was installed. Undo the deprovision in 6.5.
  • If “Deprovisioning Results” is available, but clicking it gives you a blank page, that means the deprovision was done in 6.5 before data was imported to 6.7. Undo the deprovision in 6.5.
  • If “Deprovisioning Results” is available, and clicking it gives you a lot of information (“User Account Deprovisioning”, “Exchange Mailbox Deprovisioning”, etc), then it was deprovisioned in 6.7. Undo the deprovision in 6.7.

Lessons that you can learn from Matt’s post and our hard experience:

1) Make sure you have a test environment – ideally, use snapshots of your current real data, but in a sandbox (separate from your live network).

2) You want to import the Management History using Quest’s nifty tool ONCE. That ONCE should be right before you start having the help desk and others with administrative rights on user objects use the new version of ARS for deprovisioning/undoing deprovisions.

3) Once you have decided that user object management will occur in the new version, disable Access Template Links for the help desk and others in the old version. Because the databases are separate in this scenario, a change to an Access Template Link in one version will have no effect on the other, as long as you haven’t been propogating those permissions to Active Directory (which would defeat a lot of the purpose of using ARS – only changes that happen via ARS are captured in Change History!)

It is possible to get the user objects in inconsistent Deprovision/Undo Deprovision states back to rights; however, I’m still working on that – it is going to probably involve finding the affected user objects and figuring out how to parse the ActiveRoles Management History database. Once I figure it out, naturally, I’ll let you know all about it. I will also go into more detail on how to recognize a user object with Deprovision/Undo Deprovision state issues.