2015 July

July 16, 2015

Here’s our semi-annual newsletter update on recent happenings with the UW Windows Infrastructure.

 

==== New Capabilities and Improvements ====

 

* Self-service SPNs for Application UW NetIDs. This allows an application UW NetID to set its own SPN values, just as a gMSA can, and like an OU admin can for a computer object. See https://wiki.cac.washington.edu/display/UWWI/Delegated+Service+Principal+Name+values for more details.

 

* Domain based DFS replication is now supported. This allows replication of file content across non-clustered Windows file servers. See https://wiki.cac.washington.edu/x/obv5Aw for more info.

 

* Active Directory snapshots. We now take a daily snapshot of Active Directory and retain the last 7 days. This provides an additional recovery option on top of Active Directory Recycle Bin, and daily backups of the domain controllers. This solution gives us a better recovery option for some scenarios. When an object is recovered from the recycle bin, not all attributes are recoverable, so we can augment object recovery with snapshot data. We also had an experience several years ago where a domain controller had some corruption in its local AD instance and we had to manually remove it to prevent further damage. If we had another similar experience, but corruption on a subset of all objects was replicated to other DCs, an AD snapshot would help us recover without taking the entire AD offline for an authoritative restore from backup (which would also mean some AD data loss).

 

* Monitoring improvements. We’ve made a broad investment in collecting and reporting performance data from our systems into a visual data platform called Graphite provided by the emerging Monitoring service. This platform is not restricted, so you can access that data, and might want to do so to answer questions you might be shy to ask us like “were the domain controllers really busy just now?” or “is there a problem right now with group sync?” or other questions where additional visibility on operational performance would be useful. Here’s a list of relevant UWWI performance graphs:

-ADFS Stats and Performance: https://graphs.s.uw.edu/dashboard/#services.uwwi.netid.adfs

-Group Sync Stats: https://graphs.s.uw.edu/dashboard/#GroupSync

-DC Performance including LDAP response: https://graphs.s.uw.edu/dashboard/#UWWI-DCs

-Kiwi Stats: https://graphs.s.uw.edu/dashboard/#services.uwwi.netid.kiwi

-Simple Binds: https://graphs.s.uw.edu/dashboard/#simpleBinds

 

Note that you can adjust the time period of data displayed, which can help to show interesting trends. For example, if you view the simple bind graph with the right time period, you can see the drastic effects of our efforts in the next item …

 

* Simple Bind abatement. We’ve created a reporting infrastructure that allows us to identify customers who have misconfigured their systems or application. We’ve also done the work of contacting relevant customers and reduced 99% of all simple binds. We are exploring blocking some misconfigured applications on shared systems who are difficult to identify, as well as notification to UW NetIDs whose password has been exposed.

 

====Spotlights====

 

* UWWI service staffing availability has been down over the past 6 months—this is because other UW-IT services have had higher priority work and staffing shortages. You may notice a smaller amount of new capabilities in this 6 month period, which is partially attributable to this smaller investment. We’re in the process of trying to hire someone who can backfill this staffing gap.

 

* Internal improvements. We’ve deployed a new HyperV cluster, refactored our internal documentation, and made changes to support changes to two services we depend on.

 

* We are in the midst of deploying a new security capability from Microsoft called Advanced Threat Analytics. This leverages machine learning capabilities to evaluate activity on domain controllers to identify anomalous events. This tool was acquired from a company called Aorato, and is capable of identifying pass the hash attacks and persistent “hidden” compromises of highly privileged accounts.

 

* We are also in the midst of deploying an AD-integrated certificate authority (AD Certificate Services). This is a result of exploration work we’ve mentioned in previous newsletters and will enable automatic certificate enrollment for computers in delegated OUs to support specific use cases like web servers for UW only audiences. This new infrastructure will be required to support new “Next Generation Credential” capabilities coming with Windows 10 like Windows Passport.

 

* A new governance team for Azure Active Directory has been created. This team is exploring the diverse capabilities provided by Entra ID, and guiding our direction in terms of which capabilities and configurations we enable.

 

==== Trends ====

 

* Since January, UWWI has: +9 delegated OUs (103 total), -3 trusts (53 total), +~1000 computers (10635 total), +50k users (754k total), -19k groups (108k total).

* UWWI support requests have returned to saner levels. 241 UWWI support records resolved since the last newsletter (vs. 347 in prior period and 188 in the period before that).

 

You can see metrics about UWWI at http://www.netid.washington.edu/dirinfo/stats.

 

==== What’s Next ====

 

Our objectives for the 6 months ahead include:

* Continue deployment of an AD-integrated Certificate Authority to enable a variety of multi-factor scenarios and easy internal website certificate renewal.

* Continue deployment of Advanced Threat Analytics to provide pass the hash and insight into anomalous threats

* Continue Entra ID governance team investment, with possible new objectives generated from that

* Deploy some Azure Rights Management infrastructure to support RMS pilot exploration

* ADMT 3.2 upgrade

* UW firewall GPO template to provide customers with a simple way to leverage Windows Firewall

* Explore privileged user risk mitigation–we’re interested in Microsoft’s “Just In Time” admin capability

* Explore requiring LDAP signing

* Explore providing authentication use restrictions for privileged user accounts in NETID domain

* Preferred Name (assuming this work moves forward as part of the HR/P project and has investment from other services)

* Partner with Nebula to support new Software Deployment Service via SCCM deployment in NETID (assumes depleted Nebula resourcing levels are resolved)

* Support growing Nebula migration efforts into the NETID domain

* Support Authentication service in exploring Multi-factor Authentication solutions for Windows (assumes project is launched)

* Support emerging Monitoring Service by sharing Windows expertise

 

Of the 14 forecasted objectives we listed in the last UWWI News, here’s a review on how they turned out:

  • 5 were successfully completed: Simple Bind, internal doc refactor, AD snapshots, HyperV upgrade, 3y MS tech roadmap
  • 5 were started and continue: AD-integrated CA, ATA, ADMT upgrade, SCCM exploration
  • 1 was started by dependent service, but hasn’t yet reached the point where we can start: Preferred Name

 

Note: Last summer UWWI conducted a customer survey, http://ontheroa.uservoice.com/forums/258239-uwwi. Given our current reduced staffing level, we haven’t pursued a refreshed customer survey this year, but I believe we will get some new customer input via the Entra ID governance team, and will generate a new survey when staffing permits. We will continue to use your input from that survey plus the AAD governance team to guide our investment priorities, limited by dependent service investment decisions. In some cases, lack of investment for long periods of time by dependent services may mean we choose to deploy a tactical solution instead of a strategic solution.

 

==== Your Feedback ====

 

Supporting your needs for UWWI capabilities offered via the Basic Services Bundle is our priority, so we welcome feedback on how we can make the UWWI service more valuable to you.

 

The UWWI service has a backlog or roadmap visible to customers at https://wiki.cac.washington.edu/display/UWWI/UWWI+Roadmap where you can see more details about current and some future work items.

 

You can voice your support for future objectives to help us rank priorities by voting customer surveys when we have them, ask for things that aren’t yet on our radar, or simply contact us via help@uw.edu.

 

Brian Arkills

UW-IT, UWWI Service Manager