In order to get btnx working correctly, you have to do the following in the order listed:
Install btnx
Run btnx-config.
Detect your mouse with btnx-config.
Detect your mouse buttons with btnx-config.
Configure your buttons with btnx-config.
Start btnx.
Enjoy.