It's too lean on deceleration and yes, just changing the exhaust will cause it
You can increase back-pressure, turn the idle up, adjust the fuel screw out about 1/2 turn, raise the slide needle 1 notch, increase the intake vacuum (put the top back on the air box if removed), check the air cut valve to make sure it is not damaged and is working correctly. After trying all these suggestions, learn live with it - it doesn't hurt anything, it's just annoying.