This book proposes a unified algorithmic framework based on dual optimization techniques. Adaptive algorithms based on stochastic approximation techniques are also proposed, which are shown to achieve similar performance with even much lower complexity.