We present the velocity dispersion measurements of four massive similar to 10(11) M-circle dot < 3.7 based on deep H and K-band spectra using the Keck/MOSFIRE near-infrared spectrograph. We find high velocity dispersions of order sigma(e) similar to 250 km s(-1) based on strong Balmer absorption lines, and we combine these with size measurements based on HST/WFC3 F160W imaging to infer dynamical masses. The velocity dispersions are broadly consistent with the high stellar masses and small sizes. Together with evidence for quiescent stellar populations, the spectra confirm the existence of a population of massive galaxies that formed rapidly and quenched in the early universe z > 4. Investigating the evolution at constant velocity dispersion between z similar to 3.5 and z similar to 2, we find a large increase in effective radius 0.35 +/- 0.12 dex and in dynamical-to-stellar mass ratio < log(M-dyn/M*)> of 0.33 +/- 0.08 dex, with low expected contribution from dark matter. The dynamical masses for our z similar to 3.5 sample are consistent with the stellar masses for a Chabrier initial mass function (IMF), with the ratio < log(M-dyn/M-Ch*)> = -0.13 +/- 0.10 dex suggesting an IMF lighter than Salpeter may be common for massive quiescent galaxies at z > 3. This is surprising in light of the Salpeter or heavier IMFs found for high velocity dispersion galaxies at z similar to 2 and cores of present-day ellipticals, which these galaxies are thought to evolve into. Future imaging and spectroscopic observations with resolved kinematics using the upcoming James Webb Space Telescope could rule out potential systematics from rotation and confirm these results.