This thesis contributes to the field of inverse problems with sparsity constraints. In recent years this has been a rapidly developing field within the theory of inverse and ill-posed problems. It turned out that solutions of many inverse problems have a sparse structure, which means that they can be represented using only a finite number of elements of a suitable basis or frame. To reconstruct these solutions, Tikhonov-type regularization schemes have been investigated intensively within the last years. The minimization schemes for the related Tikhonov functionals require the evaluation of the underlying operators and their adjoints. One of the main topics of this thesis is the investigation of such a minimization scheme assuming that the necessary operator evaluations are not calculated exactly, but are computed via an adaptive scheme. A second major part is the coupling of Morozov's discrepancy principle and Tikhonov regularization, where the classical quadratic penalty term has been substituted by a more general convex functional. Finally, a non-trivial inverse heat conduction problem from steel production is solved by a combination of iterated soft-shrinkage and an adaptive finite element method.