Training NER model using Stanford Core NLP CRF Classifier

Named-Entity Recognition (NER) is one of the most popular NLP tasks. It’s popular because it produces annotation result that can be used directly (eg. extracting people name from text) or indirectly (eg. extracting feature for classification task).

One of the easiest way to do it is by downloading and using latest Stanford Core NLP suite from to train a NER model using our own dataset. Here are some steps to do it.

First, prepare necessary files:

  1. Training dataset. List of word tokens annotated with their Named-Entity class (eg. PERS for Person entity). Example: jane-austen-emma-ch1
  2. Training properties. Training cofiguration properties for a CRF classifier (eg. input file(s), output file, features .etc). Example: jane-austen
  3. Test dataset. Similar to training dataset but with different list of tokens. Example: jane-austen-emma-ch2

If you have datasets in ENAMEX or Open NLP format, you can use these simple python scripts or to convert them

Enter stanfordnlp unzipped directory and run this command to train model:

java -cp "*" -prop jane-austen.prop

The result will show the output model file name ner-model.ser.gz:

[main] INFO - CRFClassifier training ... done [2.1 sec]. 
[main] INFO - Serializing classifier to ner-model.ser.gz... 
[main] INFO - done.

To test the model against the test file run this command:

java -cp "*" -loadClassifier ner-model.ser.gz -testFile jane-austen-emma-ch2.tsv

The result shows the performance of the model. In this case it achieves 82% precision, 72% recall and 77% F-measure:

[main] INFO - CRFClassifier tagged 1999 words in 1 documents at 6305.99 words per second. 
[main] INFO - Entity	P	R	F1	TP	FP	FN 
[main] INFO - PERS	0.8205	0.7273 0.7711	32	7	12 
[main] INFO - Totals	0.8205	0.7273	0.7711	32	7	12

You can also train the classifier using for language other than English as long as you provide proper training/testing dataset.

Cheers! 🍻


0 0 votes
Article Rating
Notify of
Newest Most Voted
Inline Feedbacks
View all comments