If your keyboard has a SHIFT LOCK key, you turn on capitalization by pressing it. Prior to SP4, you could NOT turn off capitalization by pressing one of the Shift keys.
After you install SP4 or greater, use regedt32 to navigate to:
to identify your <Keyboard Id>. The French keyboard layout is 0000040C. Then navigate to:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control \Keyboard Layouts\<Keyboard Id>
Select the <Keyboard Id> key and on the Edit menu, Add Value name Attributes as a type REG_DWORD and set the data value in the DWORD Editor to 0x00010000, using the Hex radix.