tokenize - Create a custom Stanford Tokenizer -



tokenize - Create a custom Stanford Tokenizer -

i involved in building natural language application should recognize entities involve machine types i.e. illustration x86 type. in add-on great if ner recognize file location, i.e. /foobar/data/ or file @ location i.e. /foobar/data/example.zip

now 1 train stanford parser arrive @ these. before creating training sample, have create labeled token file. instructions pretty clear here. had issue while tokenization. 1 can't expect stanford parser tokenize file /foobar/data/example.zip single token. want tag such file location single token , label file-location. ( asking lot if knew right entity tag should use, going through ace entity types, if can point me standard tags used machine type entities , events, helpful.)

so question :

i want customize stanford tokenizer or rather extend understand file locations or custom tokens. how can done? ( required before ner step? )

i looking @ options stanford tokenizer has here , wasn't sure of how train new tokenizer.

tokenize stanford-nlp

Comments

Popular posts from this blog

model view controller - MVC Rails Planning -

ruby on rails - Devise Logout Error in RoR -

html - Submenu setup with jquery and effect 'fold' -