Numerous real world problems can be formulated as optimization problems with various parameters to be optimized. Thus several optimization algorithms have been proposed to solve these problems. Particle Swarm Optimizer (PSO) and Differential Evolution (DE) are two relatively new optimization algorithms which have shown their strengths in the optimization world. Based on the investigation on both algorithms, this thesis presents a few improved variants of PSO and DE, which are applied to solve optimization problems with various complexities. In order to solve complex multi-modal single objective optimization, diversity enhanced technique based on the selected past solutions is proposed to discourage premature convergence of the swarm in basic PSO and Comprehensive Leaning PSO (CLPSO). Furthermore, a hybridized dynamic multi-swarm particle swarm optimizer (DMS-PSO) with the Harmony search (HS) is presented to avoid all particles getting trapped into inferior local optimal regions and to increase the diversity of the whole swarm by taking merits of the DMS-PSO and the HS. A novel two local best (lbests) based multi-objective particle swarm optimizer (MOPSO) is illustrated to solve Multi-objective Optimization problems. The 2LB-MOPSO is applied to design multiobjective robust PID controllers of two MIMO systems, as well as a Multi-objective optimization of Monopulse Antennas system. Moreover, An ensemble of -dominance external archives with the MOPSO implementation as well as an ensemble of different neighborhood sizes of the subproblems integrated with recent Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D) are demonstrated to be well performing when solving multiobjective optimization (MO) problems with different characteristics. Self-adaptive DE (SaDE) is enhanced by incorporating the JADE mutation strategy and hybridizing with modified multi-trajectory search (MMTS) algorithm (SaDE-MMTS) to solve the large scale continuous optimization problems.