Everyone has a different way of finishing CA, allot of views above are right and you may try one or all of them and not correct what you do.
However, I believe that 400 is far too coarse. I start with 1500, (lathe spinning) then go through from 3600 to 12,000 mm.(lathe stopped and spinning)
As for accelerator, why use it? (If you have of course.) more haste less speed. :wink:
What wood is it? Yes, oil does cause problems. Clean it well but it may come and bite you later. (Been there done that)