
Azure Error Message (click image for larger display)
The solution is to actually delete via the VM's area of the Windows Azure portal by selecting Disks. From Disks, you will be able to delete the VHDs that are no longer needed, as the screen shot below shows, by selecting the option to Delete Associated VHD.

Azure Delete Associated VHD (click image for larger display)