org.apache.ctakes.core.ci

Class HyphenTextModifierImpl

java.lang.Object

org.apache.ctakes.core.ci.HyphenTextModifierImpl

All Implemented Interfaces:

TextModifier

public class HyphenTextModifierImpl
extends java.lang.Object
implements TextModifier

Field Summary

Fields

Modifier and Type	Field and Description
private java.util.Map<java.lang.String,java.lang.Integer>	iv_shouldbeHyphenMap
private Tokenizer	iv_tokenizer
private int	iv_windowSize

Constructor Summary

Constructors

Constructor and Description
HyphenTextModifierImpl(java.io.InputStream hyphenfilename, int windowSize) Default constructor takes a name of the file containing hyphenated phrases, with their frequency.
HyphenTextModifierImpl(java.lang.String hyphenfilename, int windowSize)

Method Summary

Methods

Modifier and Type	Method and Description
private static boolean	applyTextModifier(TextModifier tm, java.lang.String text, java.lang.StringBuffer sb) Apply text modifier to the text TODO - move this to TextModifier and take a Logger See HyphenTextModifierImpl
private void	filterTokens(java.util.List<Token> tokenList) Filters out unwanted tokens - newlines.
static void	main(java.lang.String[] args) Simple tests of TextModification Output expected: UNSUPPORTED: TextModification with offset changes. UNSUPPORTED: TextModification with offset changes. UNSUPPORTED: TextModification with offset changes. Orig: Non Hodgkin's the x ray without any non small cell complications. New: (new text not generated, see previous messages) Non-Hodgkin Orig: 0-12 New: 0-11 x-ray Orig: 19-25 New: 18-23 non-small-cell Orig: 38-53 New: 36-50 Orig: Non Hodgkin's the x ray without any non small cell complications. New: Non-Hodgkin's the x-ray without any non-small-cell complications. Non-Hodgkin Orig: 0-11 New: 0-11 x-ray Orig: 18-23 New: 18-23 non-small-cell Orig: 36-50 New: 36-50 Note the case of the words doesn't matter.
TextModification[]	modify(java.lang.String in) Generates modifications for the specified text.
static java.util.ArrayList<java.lang.String>	test(HyphenTextModifierImpl tm, java.lang.String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

iv_shouldbeHyphenMap

private java.util.Map<java.lang.String,java.lang.Integer> iv_shouldbeHyphenMap

iv_windowSize

private int iv_windowSize

iv_tokenizer

private Tokenizer iv_tokenizer

Constructor Detail

HyphenTextModifierImpl

public HyphenTextModifierImpl(java.lang.String hyphenfilename,
                      int windowSize)

HyphenTextModifierImpl

public HyphenTextModifierImpl(java.io.InputStream hyphenfilename,
                      int windowSize)

Default constructor takes a name of the file containing hyphenated phrases, with their frequency. Currently the frequency is unused.
The case of the words in the file is unimportant - we lowercase everything when doing compares.
The file is delimited with "|" and has two fields:
hyphen-term|frequency

Method Detail

filterTokens

private void filterTokens(java.util.List<Token> tokenList)

Filters out unwanted tokens - newlines.

Parameters:

tokenList -

modify

public TextModification[] modify(java.lang.String in)
                          throws java.lang.Exception

Description copied from interface: TextModifier

Generates modifications for the specified text.

Specified by:

modify in interface TextModifier

Parameters:

in - Original document text.

Returns:

Array of TextModification objects that describe the modifications. Offset values are relative to the String object.

Throws:

java.lang.Exception

applyTextModifier

private static boolean applyTextModifier(TextModifier tm,
                        java.lang.String text,
                        java.lang.StringBuffer sb)
                                  throws java.lang.Exception

Apply text modifier to the text
TODO - move this to TextModifier and take a Logger See HyphenTextModifierImpl

Parameters:

tm - TextModifier to apply

text - Original text

sb - Buffer containing text to apply modifier to

Returns:

unableToModifyText true if modifier would require offset changes, which is not supported by this method

Throws:

java.lang.Exception

test

public static java.util.ArrayList<java.lang.String> test(HyphenTextModifierImpl tm,
                                         java.lang.String text)

main

public static void main(java.lang.String[] args)

Simple tests of TextModification
Output expected:
UNSUPPORTED: TextModification with offset changes.
UNSUPPORTED: TextModification with offset changes.
UNSUPPORTED: TextModification with offset changes.
Orig: Non Hodgkin's the x ray without any non small cell complications.
New: (new text not generated, see previous messages) Non-Hodgkin Orig: 0-12 New: 0-11
x-ray Orig: 19-25 New: 18-23
non-small-cell Orig: 38-53 New: 36-50
Orig: Non Hodgkin's the x ray without any non small cell complications.
New: Non-Hodgkin's the x-ray without any non-small-cell complications.
Non-Hodgkin Orig: 0-11 New: 0-11
x-ray Orig: 18-23 New: 18-23
non-small-cell Orig: 36-50 New: 36-50
Note the case of the words doesn't matter.

Parameters:

args - hyphen text filename (each line: hyphenated-word|freq)