tokenize - Create a custom Stanford Tokenizer -
tokenize - Create a custom Stanford Tokenizer -
i involved in building natural language application should recognize entities involve machine types i.e. illustration x86 type. in add-on great if ner recognize file location, i.e. /foobar/data/ or file @ location i.e. /foobar/data/example.zip
now 1 train stanford parser arrive @ these. before creating training sample, have create labeled token file. instructions pretty clear here. had issue while tokenization. 1 can't expect stanford parser tokenize file /foobar/data/example.zip single token. want tag such file location single token , label file-location. ( asking lot if knew right entity tag should use, going through ace entity types, if can point me standard tags used machine type entities , events, helpful.)
so question :
i want customize stanford tokenizer or rather extend understand file locations or custom tokens. how can done? ( required before ner step? )
i looking @ options stanford tokenizer has here , wasn't sure of how train new tokenizer.
tokenize stanford-nlp
Comments
Post a Comment