Sometimes the mic and/or speakers on the controller get selected as primary devices when I plug it in, forcing me to make my real mic and speakers the default again.

Run pavucontrol and disable those devices.

EDIT: Let me grab my laptop and I’ll tell you precisely where that is. One moment.

EDIT2: On the Configuration tab, for “Profile” for that device, choose “Off”.