Language can serve to constrain cognitive and emotional representations. Here, we investigate to what extent linguistic emotional information alters processing of faces with neutral expressions. Previous studies have shown that cortical processing of emotional faces differs from that of neutral faces. Electroencephalography (EEG) has revealed emotion effects for early and late event-related potentials (ERPs) such as the N1, the Early Posterior Negativity (EPN) and the Late Positive Potential (LPP). In order to study the effect of language information on face processing, 30 negative and 30 neutral descriptive phrases were presented, each followed by a neutral expression face. Participants were instructed to remember the association. We analyzed the immediate effect of information type on face processing during encoding as well as delayed effects during subsequent recognition. During encoding, faces following negative language information elicited a larger left frontal positivity between 500–700 ms after stimulus onset. During recognition, a left centro-parietal LPP was likewise increased for faces previously associated with a negative description. In addition, the parietal old/new effect was significantly increased for faces with negative information compared to new ones, while no significant effect was observed for faces with neutral information. No information effects on early perceptual ERPs (N1, EPN) were found. Reaction times (RTs) for source memory decisions (negative versus neutral) were significantly shorter for faces with negative versus neutral information. In sum, ERP results indicate that emotional significance can be linguistically induced in faces on a cortical level and, at least in an explicit memory task, this information modulates later stages of face processing and memory. Implications for cognitive effects of public media design are discussed.