Lync Server 2013 Address Book Deltas Not Being Produced – but a Fix is Coming!

UPDATE: The Cumulative Update is here, we installed it, it worked – read all about it!

We’re starting to deploy Enterprise Voice (Lync as a telephone replacement), and it’s been going reasonably well, except for one little thing that’s become a huge headache: Lync’s address book was not getting updated.

 

First attempted solution: Change Address Book full download frequency (“KeepDuration”) from 30 days to 1 day in Set-CsAddressBookConfiguration

Result: No change
Why: It would take the clients at least 30 days before they would start downloading full copies. In our case, that was probably a good thing (see “Third attempted solution”). Greig Sheridan gives a more detailed explanation of why this totally won’t work and some other things you shouldn’t bother trying.

 

Second attempted solution: Switch AddressBookAvailability in Client Policy for everyone to “WebSearchOnly”

Result: Worked like a champ for Lync 2010 and Lync 2013 clients, no change for Office Communicator 2007 R2 clients (the vast majority of non-Enterprise Voice users)
Why: Office Communicator 2007 R2 doesn’t have the ability to do Addess Book Web Queries (ABWQ). This is incorrectly stated in the main TechNet article about new features in Lync Server 2010, but correct in a similar table in this TechNet Magazine article. Grrr…

Additionally, Lync Server ClientPolicy settings do not have any effect on Office Communicator 2007 R2 clients. Those changes have to be made in the client’s Registry or via Group Policy. Since Office Communicator 2007 R2 cannot do Address Book Web Queries, there is no such Registry entry present or possible.

 

Third attempted solution (DO NOT DO THIS): Raise the MaxDeltaFileSizePercentage from the default 20 to 100.

Result:This will produce a delta file and keep it available no matter how large it is. Sites around the world that do not have superb MPLS (WAN) connections had to block all traffic from the Lync pool on Port 443 until we removed the deltas.
Why: all the OCS (and Lync) clients started downloading the deltas as soon as the users logged into them.
Why we thought this might be a good idea: Listed as a solution in the TechNet Forums thread LYNC 2013 generating Full Address book files but not the delta ones
Why it might possibly be a good idea for you: None of your sites are bandwidth-constrained, or all of your users are on their own connections
Why it is most likely not a good idea for you: The delta files can be twice the size of the “full” address book. Multiply that by the number of users at each site, and decide if that is a bandwidth hit you can take in the hour after everyone gets to work.

 

Fourth attempted solution: Made use of one of the Software Assurance (SA) support incidents we got as a Volume Licensing customer

Result: It is a known error in the Lync Server software itself, and a fix is expected in the next Cumulative Update, expected later in June or in July 2014. Hooray! We even get the incident refunded, because Microsoft agrees that it was a problem with the product itself. Keep an eye on the Lync Cumulative Updates

After loads of testing on both our side and theirs, we figured out that ABServer always tries to create deltas each day between the newest full address book (F-xxxx.lsabs) and each of the previous 29 full address book files, but, depending on the MaxDeltaFileSizePercentage, stops making the .lsabs.tmp file and deletes it. For us, the magic “keep” threshold appears to be 52; there is no difference in the delta file size, though.

There are various reports around the web of this being a problem in environments where there are more than 50,000 address book entries – environments like ours. Using ABServer -DumpFile to crack open the C-xxxx-xxxx.lsabs files created (and retained) when we had the MaxDeltaFileSizePercentage set above 52, we discovered that the first part of the file was a complete copy of the address book, while the last part was a long list of “deleted” entries. The GUIDS on the “deleted” entries list matched GUIDS in the “new” entries. The Microsoft Support engineer I’ve been working with described this as a problem comparing entries for large address books.

 

Morals of the story

  • The testing environment has to mirror your production one in data size and scope. This is a problem that does not exist in smaller environments, and, according to Microsoft Support, doesn’t affect Lync Server Standard Edition.
  • Whenever you change anything about Lync’s configuration, think about bandwidth.
  • Don’t believe everything you read on the Internet. Even on TechNet’s official product pages…
  • If you are a Volume Licensing customer (which you probably are if you’re running Lync Server on premises), make use of your Software Assurance incidents. If you don’t know whether you have any, get the person who signs your Volume Licensing agreement to sign into the Volume Licensing Center and grant someone in Operations the right to manage contacts for Software Assurance. Don’t let those incidents go to waste – your company has already paid for them!
  • Just because something is a “known defect,” it doesn’t mean that we have to live with it forever or until the next major version. Be persistent; Microsoft listens sometimes 🙂

Manage Lync Users who used to be in Domain Admins

It’s been awhile – I’ve been transitioning into Lync administration these past few months, but ARS is still a part of that…

Today, I tried changing some of my Lync user policies, and got this:

Active Directory operation failed on “DC01234.awesomedomain.com”. You cannot retry this operation: “Insufficient access rights to perform the operation 00002098: SecErr: DSID-03150BB9, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0”.You do not have the appropriate permissions to perform this operation in Active Directory. One possible cause is that the Lync Server Control Panel and Remote Windows PowerShell cannot modify users who belong to protected security groups (for example, the Domain Admins group). To manage users in the Domain Admins group, use the Lync Server Management Shell and log on using a Domain Admins account. There are other possible causes. For details, see Lync Server 2010 Help.

But… my account isn’t in Domain Admins! It was once, for about five minutes while I attempted to prove a point, but that was several months ago.

However, that is enough to mark the account to Active Directory FOREVER AND EVER as being “special”.

Dave Simm has a post that explains what happens and how to fix it: Lync enabling or making Lync changes to a user who is or was a member of the Domain Admins security group

However, one of the commenters, Rikard Strand, points out that the inheritance fix might be automatically reverted due to the fact that adminCount is still 1.

So, here’s how you fix this with the standard AD cmdlets or ARS:

1) Find the Lync-enabled users who also have adminCount=1 – this doesn’t mean that they definitely have the inheritance issue, but that they might:

ARS: Get-QADUser -SearchAttributes @{adminCount=1;'msrtcsip-PrimaryUserAddress'="*"}

AD: Get-ADUser -LDAPFilter '(adminCount=1)(msrtcsip-PrimaryUserAddress=*)'

2) Go remove them from Domain Admins or disable them for Lync if they should stay in Domain Admins – you shouldn’t be using a Domain Admin account to run your desktop applications!

3) If they’re going to remain Lync users, fix the AD object security permissions inheritance as described in Dave’s post (dsa.msc – advanced view – Security – Advanced – check the “inherit” box)

4) Finally, set the adminCount for the users you just fixed inheritance for:

ARS: Set-QADUser -Identity AWESOME\username -ObjectAttributes @{adminCount=0}

AD: Set-ADUser -Identity "cn=username,ou=SomeCity,ou=Country,dc=awesomedomain,dc=com" -Replace @{adminCount=0}