The Inner Shadow layer style in After Effects doesn't play nice with other effects. That's why I created an effects-based inner shadow preset. Use the pseudo effect controls to modify the color, distance, softness and opacity of the shadow. 

See it in action.

Install the .ffx file in your local After Effects or User Presets folder, refresh the presets list in After Effects, and have fun!

Compatible with After Effects 2021 v18.4.6 and newer.

How to install an After Effects Preset