In interactive machine learning, the learning machine is engaged in some fashion with an information source (e.g. a human or another machine). In this thesis, we study frameworks for interactive machine learning. In the first part, we consider interaction in supervised learning. The typical model of interaction in supervised learning has been restricted to labels alone. We study a framework in which the learning machine can receive feedback that goes beyond labels of data points, to features that may be indicative of a particular label. We call this framework learning with feature feedback and study it formally in several settings.