This thesis demonstrates techniques that provide faster and more accurate solutions to a variety of problems in machine learning and signal processing. The author proposes a "greedy" algorithm, deriving sparse solutions with guarantees...