I start by presenting the problem dealt with, what is named entity recognition (NER) and why it is relevant, then I describe what we did to evaluate the systems which do NER: the HAREM contest. In this connection, I will stress what was new compared to other evaluation contests and will put the emphasis on methodological questions related to natural language processing (NLP). Several of the the evaluation choices will be discussed and motivated. In the talk, I also intend to give a brief overview of Linguateca and of the evaluation contest paradigm in NLP, and end with a discussion of remaining challenges (in NER and evaluation contests in general). HAREM stands for "HAREM - Avaliação de Reconhecimento de Entidades Mencionadas".