Reading notes in books and other printed matter are of increasing interest in Philology and Cultural History. However, we still lack an understanding of their epistemic foundations. With reference to Thomas Mann’s private library, I suggest viewing the act of annotating with pens itself as an epistemic practice. For this, I introduce the term ‘pen traces’ for all phenomena left behind by pens, and distinguish between four different forms of knowledge that influence their emergence: the knowledge of readers, the knowledge of authors, textual knowledge and contextual knowledge. By shedding light on pen traces as traces of an epistemic practice, finally, I point out the need for a practice theory of annotation.