The estimation of parameters that affect the dynamics of objects—such as viscosity or internal degrees of freedom—is an important step in autonomous and dexterous robotic manipulation of objects. However, accurate and efficient estimation of these object parameters may be challenging due to complex, highly nonlinear underlying physical processes. To improve on the quality of otherwise hand-crafted solutions, automatic generation of control strategies can be helpful. We present a framework that uses active learning to help with sequential gathering of data samples, using informationtheoretic criteria to find the optimal actions to perform at each time step. We demonstrate the usefulness of our approach on a robotic hand-arm setup, where the task involves shaking bottles of different liquids in order to determine the liquid’s viscosity from only tactile feedback. We optimize the shaking frequency and the rotation angle of shaking in an online manner in order to speed up convergence of estimates.