The development of competent AI for real-time strategy games such as StarCraft is made difficult by the myriad of strategic and tactical reasonings which must be performed concurrently. A significant portion of StarCraft gameplay is in managing tactical conflict with opposing forces. We present a modular framework for simulating AI vs. AI conflicts through an XML specification, whereby the behavioural and tactical components for each force can be varied. Evolutionary computation can be employed on aspects of the scenario to yield superior solutions. Through evolution, a StarCraft AI tournament bot achieved a success rate of 68% against its original version. We also demonstrate the use of evolutionary computation to yield a tactical attack path to maximise enemy casualties. We believe that our framework can be used to perform automatic refinement on AI bots in StarCraft.