The subject behavior may occur if the Windows DNS server hosts an _msdcs.ForrestRootName zone with more than ~ 400 GC SRV records, and in other scenarios with other record types.
NOTE: If the client queries a DNS server that forwards the query to another DNS server that hosts the zone, that server sends a SERVER FAILURE message to the client.
When a client requests a resource record set that is larger than the UDP maximum message size, which is currently 512 bytes, the client is instructed to switch to TCP for the query. Using TCP, the server uses a fixed buffer size, which may truncate the response.
If you experience this problem because of the number of active global catalog servers, the only workaround is to decrease the number of records in the resource record set. To accomplish this, unregister some global catalog servers in DNS by setting the DnsAvoidRegisterRecords Value Name on the global catalog servers that you want to unregister. Set this REG_MULTI_SZ data type to:
Gc GcIpAddress GenericGc
NOTE: See the following Microsoft Knowledge Base articles: