Simply grab and pull. A drop of 3 in 1 oil into where it is being held, will be helpful. If the wheels pull off and the stem still remains in the chair base, try another method.
Recommend method- You'll need 1 of these 3 tools Pry bar Handy bar CURVED wire cutter but not as good as the others Above all, do not use a SCREWDRIVER it makes a bloody mess After working the tool lifting edge under the caster, pull down to create leverage to lift caster upwards. Add shims as or if needed until caster is out of base. One drop of oil where the caster stem is in the base is very helpful.
Prying-With a screwdriver, twist to raise the stem only slightly enough to insert the nail pulling part of the hammer (see below). Using the screwdriver only to remove the entire caster will result in your stabbing yourself and making a hole in your hand. It hurts!
Prying-This method may work. Use the nail puller end of a claw hammer. Place the nail pulling open end between the ridge on the castor stem and the chair base. Locate a leverage point to pry down onto and pull the hammer handle downward. Especially if you used a drop of oil, the stem should rise up a small amount providing an even better bite of the stem. When partly removed, you may need to place a thin piece of wood on top of the leverage point to press the hammer against for the stem to remove entirely.