This thesis discusses the main issues of human activity recognition systems, including automatic human activity segmentation, non-meaningful activity rejection and multi-agent activity recognition, and presents the contribution of this project for these issues. Three contributions are presented in this thesis. Firstly, a background-state based auto-segmentation framework is proposed to segment human activities of interest from continuous input. Secondly, the non-meaningful activities is rejected be a pairwise likelihood ratio test (PLRT), which has a good performance while only relying on information of meaningful patterns. Thirdly, an observation decomposed hidden Markov model (ODHMM) is proposed to recognize multi-agent activities, where the role of each agent can be identified automatically. These contributions concerned on various important aspects of human activity recognition and make it possible to build a real-life system.