Visual tracking and surveillance has become an active research area of computer vision due to the demand from the public for improved security and safety. The research work of visual tracking and surveillance involves many technical issues, such as motion segmentation, object representation, object tracking and behavior understanding. Among these issues, object representation and tracking are specially important. Being able to maintain tracking of objects in video sequences is not only useful by itself but also a crucial step to higher level video interpretation. The problems are made difficult due to non-stationary environment, persistent and temporary occlusion of multiple interacting objects and low image resolution in cases of distant viewing, which occur frequently in real applications. A stochastic transductive adaptation method is proposed in this thesis to address the problem of non-stationary object tracking in a complex environment. The proposed stochastic transductive adaptation algorithm combines stochastic transductive learning with a locally exploring particle filter. This adaptive tracker can efficiently and successfully handle on-rigid objects under different appearance changes by its stochastic transductive learning ability. Objects can be tracked well despite severe occlusion or clutter.