The NIRSpec instrument for the James Webb Space Telescope (JWST) is a highly versatile near-infrared spectrograph that can be operated in various observing modes, slit apertures, and spectral resolutions. Obtaining dedicated calibration data for all possible combinations of aperture and disperser is an intractable task. We have therefore developed a procedure to derive a highly realistic model of the instrument's optical geometry across the entire field of view, using calibration data acquired through only a subset of NIRSpec apertures, which nevertheless allows the light paths within the spectrograph to be accurately computed for all apertures and all observing modes. This parametric instrument model thus provides the basis for the extraction of wavelength-calibrated spectra from any NIRSpec exposure, regardless of observing mode or aperture used. Optimizing the NIRSpec instrument model and deriving its final wavelength and astrometric calibration was one of the most crucial elements of the NIRSpec commissioning phase. Here, we describe the process of re-fitting the NIRSpec instrument model with in-orbit commissioning data, and present its final performance in terms of wavelength accuracy and astrometric calibration.