This thesis is dedicated to the methods of quantum noise reduction in composite quantum systems. Thequantum noise is a major limitation for the sensitivity of future generations of laser interferometric Gravitational Wave Detectors (GWDs), whose mirrors can be regarded as free test masses. One of the most promising directions in quantum noise suppression for such GWDs is the application of non-classical states of light. In order to boost the sensitivity of GWD in broadband spectral range, it has been proposed to use frequency dependent single-mode squeezed states or conditional two-mode squeezed states. Theoretically, the performance of these protocols will be only restricted by the quality of the non-classical states of light. However, the practical implementation of such schemes involves the utilization of so-called filter optical cavities. An external resonator with extremely narrow bandwidth can be used to set the desired dependence of squeezing angle on the Fourier frequency for the singlemode squeezing. If two-mode squeezed states are exploited, the GWD itself can play a role of the filtering cavity.In turn, it imposes strict requirements on tuning the multiple interlinked parameters of the optical interferometer.Both approaches are complicated by experimental challenges related to the control of filtering resonators and/or changes in the established configuration of GWDs.This thesis is dedicated to the methods of quantum noise reduction in composite quantum systems. Thequantum noise is a major limitation for the sensitivity of future generations of laser interferometric Gravitational Wave Detectors (GWDs), whose mirrors can be regarded as free test masses. One of the most promising directions in quantum noise suppression for such GWDs is the application of non-classical states of light. In order to boost the sensitivity of GWD in broadband spectral range, it has been proposed to use frequency dependent single-mode squeezed states or conditional two-mode squeezed states. Theoretically, the performance of these protocols will be only restricted by the quality of the non-classical states of light. However, the practical implementation of such schemes involves the utilization of so-called filter optical cavities. An external resonator with extremely narrow bandwidth can be used to set the desired dependence of squeezing angle on the Fourier frequency for the singlemode squeezing. If two-mode squeezed states are exploited, the GWD itself can play a role of the filtering cavity.In turn, it imposes strict requirements on tuning the multiple interlinked parameters of the optical interferometer.Both approaches are complicated by experimental challenges related to the control of filtering resonators and/or changes in the established configuration of GWDs.

The main focus of this thesis is the investigation of the alternative strategy, where quantum noise reduction will be achieved by using an auxiliary harmonic oscillator that is the ensemble of spin polarized cesium atoms.Analogous to GWD, the spin oscillator is also exposed to quantum noise. However, the total noise in the hybrid system, composed of an atomic ensemble and a GWD, can be partially or completely eliminated. It has been previously demonstrated that the reference spin oscillator facilitates mitigation of the quantum noise in a dielectric membrane in MHz spectral range. At the same time, the atomic oscillator is highly tunable and can in principle be adjusted to match the GWD, making a joint measurement of two disparate systems feasible. From an experimental point of view, the spin oscillator might be added to the GWD as an external module without necessity to substantially modify the current design of the detection schemes. On the other hand, the spin ensembles compare favorably with filtering cavities in terms of complexity of control.
The dynamics of a spin oscillator, suitable for broadband sensitivity enhancement of GWDs in the spectral range of interest, should be predominantly driven by quantum noise at acoustic spectral frequencies. Moreover, in order to set the link between the GWD and the atomic ensemble, an Einstein–Podolsky–Rosen (EPR)-entangled state of light resilient to low-frequency technical noise is needed. In this thesis we report the preparation and characterization of each subsystem, performed separately. First, we present the entanglement source of two optical modes, having the wavelengths of 852 nm and 1064 nm that match the frequencies of lasers used to probe the atomic spin ensemble and GWD, respectively. We observe EPR correlations spanning down to audioband (Ω ≳ 10 kHz). The entanglement is verified by EPR-steering and Duan criteria. The recorded level of two mode squeezing (−7.1 dB) is one of the highest (to our knowledge) to date for such large wavelength separation. Secondly, we prove the strong contribution of quantum back action (QBA) noise to the dynamics of the spin oscillator, using ponderomotive squeezing as a benchmark. In particular, we measure −3 dB and −0.7 dB of light noise reduction below shot noise level caused by interaction with the atomic ensemble, whose resonance frequency is set to 20 kHz and 6 kHz, respectively. The factors compromising the spin oscillator in quantum regime at near-DC spectral frequencies are explored and discussed. It has also been shown that QBA-dominated motion can be obtained for the atomic ensemble initialized in a state with an effective negative mass, which is a key feature for quantum noise cancellation in the joint measurement. The dissertation is complemented by theoretical analysis of the scheme under realistic experimental conditions.
The achieved performance of two quantum systems makes them ready for proof-of-principle experiment addressing the upper part of audio frequency range. In a long term prospective, presented results can serve as a foundation for quantum noise reduction in sensitivity band of contemporary state-of-the-art interferometric GWDs. The comprehensive mathematical model of the experiment accounts for various imperfections and yields the set of parameters to optimize the efficiency of the protocol.

The main focus of this thesis is the investigation of the alternative strategy, where quantum noise reduction will be achieved by using an auxiliary harmonic oscillator that is the ensemble of spin polarized cesium atoms.Analogous to GWD, the spin oscillator is also exposed to quantum noise. However, the total noise in the hybrid system, composed of an atomic ensemble and a GWD, can be partially or completely eliminated. It has been previously demonstrated that the reference spin oscillator facilitates mitigation of the quantum noise in a dielectric membrane in MHz spectral range. At the same time, the atomic oscillator is highly tunable and can in principle be adjusted to match the GWD, making a joint measurement of two disparate systems feasible. From an experimental point of view, the spin oscillator might be added to the GWD as an external module without necessity to substantially modify the current design of the detection schemes. On the other hand, the spin ensembles compare favorably with filtering cavities in terms of complexity of control.The main focus of this thesis is the investigation of the alternative strategy, where quantum noise reduction will be achieved by using an auxiliary harmonic oscillator that is the ensemble of spin polarized cesium atoms.Analogous to GWD, the spin oscillator is also exposed to quantum noise. However, the total noise in the hybrid system, composed of an atomic ensemble and a GWD, can be partially or completely eliminated. It has been previously demonstrated that the reference spin oscillator facilitates mitigation of the quantum noise in a dielectric membrane in MHz spectral range. At the same time, the atomic oscillator is highly tunable and can in principle be adjusted to match the GWD, making a joint measurement of two disparate systems feasible. From an experimental point of view, the spin oscillator might be added to the GWD as an external module without necessity to substantially modify the current design of the detection schemes. On the other hand, the spin ensembles compare favorably with filtering cavities in terms of complexity of control.The main focus of this thesis is the investigation of the alternative strategy, where quantum noise reduction will be achieved by using an auxiliary harmonic oscillator that is the ensemble of spin polarized cesium atoms.Analogous to GWD, the spin oscillator is also exposed to quantum noise. However, the total noise in the hybrid system, composed of an atomic ensemble and a GWD, can be partially or completely eliminated. It has been previously demonstrated that the reference spin oscillator facilitates mitigation of the quantum noise in a dielectric membrane in MHz spectral range. At the same time, the atomic oscillator is highly tunable and can in principle be adjusted to match the GWD, making a joint measurement of two disparate systems feasible. From an experimental point of view, the spin oscillator might be added to the GWD as an external module without necessity to substantially modify the current design of the detection schemes. On the other hand, the spin ensembles compare favorably with filtering cavities in terms of complexity of control.