Dissipation dilution enables extremely low linear loss in stressed, high aspect ratio nanomechanical resonators, such as strings or membranes. Here, we report on the observation and theoretical modeling of nonlinear dissipation in such structures. We introduce an analytical model based on von Karrnan theory, which can be numerically evaluated using finite-clement models for arbitrary geometries. We use this approach to predict nonlinear loss and (Duffing) frequency shift in ultracoherent phononic membrane resonators. A set of systematic measurements with silicon nitride membranes shows good agreement with the model for low-order soft-clamped modes. Our analysis also reveals quantitative connections between these nonlinearities and dissipation dilution. This is of interest for future device design and can provide important insight when diagnosing the performance of dissipation dilution in an experimental setting.