The optical Stark effect is a tell-tale signature of coherent light-matter interaction in excitonic systems, wherein an irradiating light beam tunes exciton transition frequencies. Here we show that, when excitons are placed in a nanophotonic cavity, the excitonic Stark effect can become highly nonlinear, exhibiting multivalued and hysteretic Stark shifts that depend on the history of the irradiating light. This multistable Stark effect (MSE) arises from feedback between the cavity mode occupation and excitonic population mediated by the Stark-induced mutual tuning of the cavity and excitonic resonances. Strikingly, the MSE manifests even for very dilute exciton concentrations and can yield discontinuous Stark shift jumps of order meV. We expect that the MSE can be realized in readily available transition metal dichalcogenide excitonic systems placed in planar photonic cavities at modest pump intensities. This phenomenon can provide new means to engineer coupled states of light and matter that can persist even in the single exciton limit.