edu.mit.jmwe.data

Interface IToken

All Superinterfaces:

IHasForm

All Known Subinterfaces:

IConcordanceToken

All Known Implementing Classes:

ConcordanceTagger.TaggerToken, ConcordanceToken, Token

public interface IToken
extends IHasForm

A token that is a constituent of an sentence. They may represent single runs of non-whitespace characters (not containing whitespace) or punctuation. Tokens may or may not be tagged, and may or may not already have stems assigned.

Since:

jMWE 1.0.0

Version:

$Id: IToken.java 285 2011-05-05 19:44:16Z markaf $

Author:

Nidhi Kulkarni, M.A. Finlayson

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	getStems() Returns an unmodifiable list of stems, all in lowercase.
java.lang.String	getTag() Returns the part of speech tag for this token, or null if the token is not tagged.

Methods inherited from interface edu.mit.jmwe.data.IHasForm

getForm

Method Detail

getTag

java.lang.String getTag()

Returns the part of speech tag for this token, or null if the token is not tagged. If the part of speech is null, no part of speech has yet been assigned.

Returns:

the part of speech tag for this token, or null if the token is not tagged.

Since:

jMWE 1.0.0

getStems

java.util.List<java.lang.String> getStems()

Returns an unmodifiable list of stems, all in lowercase. The order of the stems depends on the implementation. No stem should be repeated in the list. If the method returns an empty list, this means that the token is not stemmable. If the method returns null, this means no stemming has yet been attempted.

Returns:

a possibly null, possibly empty list of lowercase stems

Since:

jMWE 1.0.0