Misfiring is caused by the spark (lack of, strength, time) - the 400EX does need a battery to run - it is an AC excited ignition - the battery is just there for the lights and starter motor.
Cam timing and valve lift have little to do with misfiring - in your list of 'claimed' new parts only the stator, CDI unit, coil and spark plug have anything to do with the problem you are experiencing. On part that was not mentioned was the spark plug cap. I have seen the plug cap fail and it is seldom suspect by novice and seasoned mechanics alike. Now, the spark plug is not new! As soon as an engine is started, run and shut off the spark plug is now a used plug.
The fuel air mixture could cause misfiring if it's over rich and fouling the plug. Having to hold the throttle open is a method of leaning the mixture (it's how you start a flooded engine). If the fuel level in the float bowl is too high, the engine can run over rich regardless of the jet sizes. The pilot jet is what the engine idles on. The farther out you turn the pilot screw, the richer the mixture becomes. The POS K&N filter will definitely lean the mixture at high RPM and the pipe will have a minor effect on jetting at higher throttle openings, but the idle is only slightly affected. I would be satisfied with 38 and 150 to 160 jetting or maybe bump the pilot jet up to a 40 and set the pilot screw at 1/2 to one turn out. From your description, Is there any possibility the pilot jet was drilled out? I think I would try a new 38 pilot jet and connect a timing light to the spark plug just to verify spark - if the spark is intermittent and the timing light quits flashing and when it does flash it backfires, that just confirms an ignition problem.
The engine will run with the cam one tooth either way from correct - if the cam is off by 1/2 tooth, it indicates the chain is worn and needs replaced. If the cam is one tooth off either way, the effect is; one way it is hard to start, doesn't idle well, but runs like a raped ape at high RPM - off the other way it starts easier, idles exceeding well, but the power and high RPM performance suffer - you are looking for that happy medium.
Get a few new spark plugs, test the spark plug cap, test the wiring connections in the wiring harness and eliminate the kill circuit as a suspect (disconnect the black wire with the white stripe at the CDI box) if it runs with the b/w wire disconnected, start the engine, disconnect the key switch and handlebar kill switch, run it with the b/w wire connected and it is running right then plug in each kill switch one at a time to see if it affects the spark - if it does, replace the offending switch - also check all ground connections including motor mount bolts and see where that get you.
Concentrate on a spark issue before worrying about a fuel issue - got to get it idling before worrying about high speed running.
Almost forgot - check the oil to verify it is not half gas - if the machine is parked overnight with the fuel turned on, the oil can become contaminated with gas and it will run bad - gets worse as it warms due to the heat evaporating gas out of the crankcase which vents into the air box.
Cam timing and valve lift have little to do with misfiring - in your list of 'claimed' new parts only the stator, CDI unit, coil and spark plug have anything to do with the problem you are experiencing. On part that was not mentioned was the spark plug cap. I have seen the plug cap fail and it is seldom suspect by novice and seasoned mechanics alike. Now, the spark plug is not new! As soon as an engine is started, run and shut off the spark plug is now a used plug.
The fuel air mixture could cause misfiring if it's over rich and fouling the plug. Having to hold the throttle open is a method of leaning the mixture (it's how you start a flooded engine). If the fuel level in the float bowl is too high, the engine can run over rich regardless of the jet sizes. The pilot jet is what the engine idles on. The farther out you turn the pilot screw, the richer the mixture becomes. The POS K&N filter will definitely lean the mixture at high RPM and the pipe will have a minor effect on jetting at higher throttle openings, but the idle is only slightly affected. I would be satisfied with 38 and 150 to 160 jetting or maybe bump the pilot jet up to a 40 and set the pilot screw at 1/2 to one turn out. From your description, Is there any possibility the pilot jet was drilled out? I think I would try a new 38 pilot jet and connect a timing light to the spark plug just to verify spark - if the spark is intermittent and the timing light quits flashing and when it does flash it backfires, that just confirms an ignition problem.
The engine will run with the cam one tooth either way from correct - if the cam is off by 1/2 tooth, it indicates the chain is worn and needs replaced. If the cam is one tooth off either way, the effect is; one way it is hard to start, doesn't idle well, but runs like a raped ape at high RPM - off the other way it starts easier, idles exceeding well, but the power and high RPM performance suffer - you are looking for that happy medium.
Get a few new spark plugs, test the spark plug cap, test the wiring connections in the wiring harness and eliminate the kill circuit as a suspect (disconnect the black wire with the white stripe at the CDI box) if it runs with the b/w wire disconnected, start the engine, disconnect the key switch and handlebar kill switch, run it with the b/w wire connected and it is running right then plug in each kill switch one at a time to see if it affects the spark - if it does, replace the offending switch - also check all ground connections including motor mount bolts and see where that get you.
Concentrate on a spark issue before worrying about a fuel issue - got to get it idling before worrying about high speed running.
Almost forgot - check the oil to verify it is not half gas - if the machine is parked overnight with the fuel turned on, the oil can become contaminated with gas and it will run bad - gets worse as it warms due to the heat evaporating gas out of the crankcase which vents into the air box.