Our network server environment contains a mixture of Windows 2000, Windows NT 4.0, SCO Group's SCO OpenServer, and Linux Red Hat 7.x servers, and we use NT Server 4.0 for our DHCP needs. I've been tasked with finding a DHCP server that has the capability to lease address information only to clients with reservations. The only way I can think of to achieve this result is to have only reserved addresses in the available scope. But with that approach, how could we remove a reservation from midscope when an employee leaves the company or no longer has a need for DHCP? The goal would be to secure DHCP so that it leases addresses only to clients with reservations; any machines that we haven't assigned a static address to or configured for DHCP would be unable to connect to the network. This restriction is important because we have four remote locations (Australia, the Netherlands, Puerto Rico, and another in the United States) and each site sets up its own computers, but we (as the main office) have control over the networking configuration (e.g., DNS, DHCP, router rules). We want to ensure that the other offices aren't inappropriately connecting machines to the network. Additionally, this setup would eliminate the possibility of Windows 9x machines resolving a DHCP address because someone has tampered with network settings and removed the static addressing.
Your proposed method for securing your DHCP server would be a weak control because users could determine unused IP addresses within the subnet. A better solution is to use either IP Security (IPSec) or 802.1x, which isn't just a wireless protocol; you can also use 802.1x to lock down access to your WAN. If you use IPSec, you must configure all your authorized computers to require IPSec Authenticated Header (AH) mode before establishing any connections to other computers. AH mode requires certificate, secret key, or Kerberos authentication and performs integrity checking and spoof prevention on every packet. (Encapsulating Security Payload—ESP—mode provides data encryption in addition to AH's features.) With this method, if someone connects an unauthorized computer to your network, the unauthorized machine wouldn't be able to communicate with any other computers on the network. You might see some performance degradation on servers because they must handle IPSec-related hashing and encryption for many client connections. Also, you might have problems getting IPSec support on non-Windows computers or integration problems for systems that do support IPSec.
The other technology, 802.1x, was originally designed to control access to wired networks, although it has gained acceptance through the push to secure Wi-Fi networks. The 802.1x technology uses port-based access control and Remote Authentication Dial-in User Service (RADIUS) to ensure that only authorized computers can use a network. However, using 802.1x requires new hardware and client software, and support for the technology on wired networks is uncertain. You might look at third-party DHCP solutions, such as those from MetaInfo, which provide security, auditing, and management for IP networks.
Although the client-reservations approach you describe would provide a measure of control, it wouldn't defeat savvy users and would be labor-intensive to administer. Another alternative would be to write a script that regularly scans your DHCP server's log for leases to unauthorized media access control (MAC) addresses and alerts you through email.