The subject behavior is caused by the Recycle Bin, which doesn't understand mounted volumes.
To workaround this issue, use either of the following:
- In Windows Explorer, Hold down the SHIFT key before your press Delete. This bypasses the Recycle Bin.
- Delete the folder from a CMD prompt, use the RD /S /q FolderPath command.