This work is a fundamental study on the model-based prediction and control of vibration in nonlinear rotordynamic systems. The work has potential applications in the design of rotating machines such as gas turbines, compressors, turbo-pumps and turbo-generators. Such machines are designed so that unwanted interaction between rotating and stationary parts is normally avoided. However, under abnormal operating conditions, excessive vibration of the machine structure can lead to contact between the rotor and its surround. This can cause the machine vibration to become dynamically unstable, which can result in failure or damaging of the machine. The two main forms of vibrational instability considered in this study are: 1. Amplitude jump in synchronous whirl vibration 2. Friction-driven backward whirl instability Modeling of this behaviour can be made based on the assumption that the machine structure has linear elastic behaviour and that nonlinear interaction between the rotor and its surround can be described by a compliant contact model that accounts for normal and tangential components of rotor-stator interaction forces. Based on the proposed model, a method is _rst developed to predict rotor synchronous vibration under conditions of full circular rub between a rotor and its surround. The method can be used to determine the operating conditions under which jumps in behaviour from contact-free whirl to a state involving persistent contact with the surround are possible. Comparison between theoretical predictions and behaviour of an experimental _exible rotor test rig are given. Further work focuses on the development of vibration control strategies for preventing vibrational instabilities caused by rotor-stator contact interaction. This work draws on a state-space analysis approach developed previously by the author using Lyapunov's direct method of stability analysis. The approach is extended here for application to vibration controller design problems. The work addresses both mentioned types of vibrational instability and develops a number of control methods based on feedback of measured rotor-stator interaction forces that are capable of preventing unstable vibration behaviour. The control methods are compared through analysis and simulation. Experimental results from the exible rotor test rig are also given that involve feedback control of an active magnetic bearing. It is shown that, with an appropriate controller design, the control forces applied to the rotor can be e_ective, not only in preventing instability, but also in minimizing rotor-stator contact forces. At the time of writing, test are ongoing to investigate a wider range of operating conditions and thereby verify the full potential of the proposed methods.