If you try and delete a virtual hard disk (VHD) that was part of a virtual machine (VM), even though the VM has been deleted, you will get an error that the lease on the BLOB hasn't expired, which the screen shot shows below.
Azure Error Message (click image for larger display)
The solution is to actually delete via the VM's area of the Windows Azure portal by selecting Disks. From Disks, you will be able to delete the VHDs that are no longer needed, as the screen shot below shows, by selecting the option to Delete Associated VHD.
Azure Delete Associated VHD (click image for larger display)