Eye movements during real-world scene viewing have been shown to be influenced by two main mechanisms of attention allocation: bottom-up and top-down. The exact effect each mechanism has on attention allocation is the focus of much debate, however, the use of eye-trackers has allowed progress towards a more comprehensive understanding. Here we utilise the Itti and Koch (2000) model of saliency-based search to manipulate a critical object within forty-eight real-world images. In doing so, we aim to analyse the effect of visual saliency on attention allocation during real-world scene viewing and active search. To examine the effect of visual saliency and task demands on attention allocation we recorded the time taken to first fixate on a critical object within each scene, as well as first fixation durations and average saccade amplitudes. In addition, semantic congruence of the critical object was manipulated in 50% of the real-world images. Results from two experiments indicate that task demands significantly influence time taken to first fixate on the critical object. Contrary to previous research, it was observed that attention allocation during real-world scene viewing is guided predominately by higher level cognitive factors. Conversely, attention allocation during active search was influenced predominately by low-level visual features. Interactions of saliency and semantic congruence were also observed. Together these results suggest that a single attention allocation mechanism is not sufficient to explain the diversity of eye movement patterns recorded and that additional research is required to fully understand the factors influencing attention allocation during real-world scene viewing and active search.