I fly LOP behind one of those large turbocharged Continental engines in a different airplane (used to have a couple of Commanders). I cruise LOP all the time, but I had the benefit of others knowing roughly the settings to use as a starting point, a MAP controller, fuel injectors, and full instrumentation. My plane has GAMI injectors and runs smoothly down to over 100 degrees LOP.
If I were to try it on an engine without any idea of the LOP flows, I would first get to an altitude such as 10,000' and set the power to 60-65% ROP at normal-to-high cruise RPM. I'd let the speed settle in and note the IAS.
Since the APS folks say you can't hurt an engine with the mixture knob if you're <= 65% power, I'd lean the mixture to peak reasonably quickly, note the peak TIT, and keep going to the lean side. Assuming the engine continues to run acceptably past peak, you will see the TIT come back down from peak as you lean and the CHTs will start dropping, too. You want the CHTs below 380 degrees.
Once you're on the lean side, if you're turbo you can give an inch or so more MAP and the TIT and CHTs will decrease at the same fuel flow -- you can use this to drop TIT and CHT if needed and/or it lets you feed more fuel for more power and stay LOP. After all this tweaking if you achieve the same IAS as above, then you know you're operating at 60-65% LOP. Record the RPM, MAP, and fuel flow and you should be able to get to these LOP settings much quicker next time. I wouldn't go above 60% power if I didn't know I have balanced fuel flows and/or don't have all cylinder CHT monitoring.
If you're carbureted and not fuel injected, i understand that adding some amount of carb heat can sometimes help the engine run more smoothly LOP. I've never tried this. I'd also be careful with a turbo and no MAP controller because altitude changes might change your mixture.
The GAMI site has a "GAMI lean test" procedure you can use to see how balanced your fuel delivery system is. The APS folks at
www.advancedpilot.com have a good online course for LOP operations.
I will confess to having gone through 4 cylinders on my Continental engine in 800 hours. "The net" says these engines consume cylinders no matter how one runs them....and I'm inclined to believe it. However, I've always run this engine LOP and obviously don't know what the results would have been had I always run ROP. I will say the engine temperatures are considerably lower LOP and the oil, plugs, cylinders, valves all stay quite clean. It's a thirsty engine; my fuel flows are around 3GPH lower LOP in cruise.