org.apache.ctakes.ytex.kernel

Class BaseSparseDataFormatter

java.lang.Object

org.apache.ctakes.ytex.kernel.BaseSparseDataFormatter

All Implemented Interfaces:

SparseDataFormatter

Direct Known Subclasses:

LibSVMFormatterFactory.LibSVMFormatter, WekaFormatterFactory.WekaFormatter

public abstract class BaseSparseDataFormatter
extends Object
implements SparseDataFormatter

Field Summary

Fields

Modifier and Type	Field and Description
protected Properties	exportProperties export properties - properties file that controls what to do for this export
protected KernelUtil	kernelUtil
protected Map<String,com.google.common.collect.BiMap<String,Integer>>	labelToClassIndexMap map of label - [class name - class index]
protected int	maxAttributeIndex 1-based attribute index
protected Map<String,Map<String,Integer>>	nominalAttributeMap map of nominal attribute - [nominal attribute value - attribute index].
protected Map<String,Integer>	numericAttributeMap map of numeric attribute - attribute index.
protected String	outdir directory to export files to, with trailing separator added on if necessary

Fields inherited from interface org.apache.ctakes.ytex.kernel.SparseDataFormatter

ATTR_INSTANCE_ID, SCOPE, SCOPE_FOLD, SCOPE_LABEL

Constructor Summary

Constructors

Constructor and Description
BaseSparseDataFormatter(KernelUtil kernelUtil)

Method Summary

Modifier and Type	Method and Description
protected void	addNumericAttribute(BufferedWriter w, String attributeName)
protected int	exportAttributeNames(BufferedWriter w, SparseData sparseData) assign indices to each attribute.
protected void	exportAttributeNames(SparseData sparseData, String label, Integer run, Integer fold)
protected void	exportSparseMatrix(String filename, SparseData sparseData) export sparse matrix data for use in matlab/R.
protected void	exportSparseRow(SparseData bagOfWordsData, long instanceId, BufferedWriter wData, int row)
protected SortedMap<Integer,Double>	getSparseLineValues(SparseData bagOfWordsData, Map<String,Integer> numericAttributeMap, Map<String,Map<String,Integer>> nominalAttributeMap, long instanceId) create a map of attribute index - attribute value for the given instance.
void	initializeExport(InstanceData instanceLabel, Properties properties, SparseData sparseData) get needed properties out of outdir.
protected void	updateLabelClassMapTransductive() add the 'unlabeled' class id to the classIndexMap if it isn't there already

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.ctakes.ytex.kernel.SparseDataFormatter

clearFold, clearLabel, exportFold, initializeFold, initializeLabel

Field Detail

kernelUtil

protected KernelUtil kernelUtil

outdir

protected String outdir

directory to export files to, with trailing separator added on if necessary

numericAttributeMap

protected Map<String,Integer> numericAttributeMap

map of numeric attribute - attribute index.

nominalAttributeMap

protected Map<String,Map<String,Integer>> nominalAttributeMap

map of nominal attribute - [nominal attribute value - attribute index].

labelToClassIndexMap

protected Map<String,com.google.common.collect.BiMap<String,Integer>> labelToClassIndexMap

map of label - [class name - class index]

maxAttributeIndex

protected int maxAttributeIndex

1-based attribute index

exportProperties

protected Properties exportProperties

export properties - properties file that controls what to do for this export

Constructor Detail

BaseSparseDataFormatter

public BaseSparseDataFormatter(KernelUtil kernelUtil)

Method Detail

exportAttributeNames

protected void exportAttributeNames(SparseData sparseData,
                                    String label,
                                    Integer run,
                                    Integer fold)
                             throws IOException

Throws:

IOException

exportAttributeNames

protected int exportAttributeNames(BufferedWriter w,
                                   SparseData sparseData)
                            throws IOException

assign indices to each attribute.

Parameters:

outdir - directory to write file to

sparseData -

numericAttributeMap -

nominalAttributeMap - for nominal indices, create an index for each value.

Throws:

IOException

addNumericAttribute

protected void addNumericAttribute(BufferedWriter w,
                                   String attributeName)
                            throws IOException

Throws:

IOException

getSparseLineValues

protected SortedMap<Integer,Double> getSparseLineValues(SparseData bagOfWordsData,
                                                        Map<String,Integer> numericAttributeMap,
                                                        Map<String,Map<String,Integer>> nominalAttributeMap,
                                                        long instanceId)

create a map of attribute index - attribute value for the given instance.

Parameters:

bagOfWordsData -

numericAttributeMap -

nominalAttributeMap -

instanceId -

Returns:

exportSparseRow

protected void exportSparseRow(SparseData bagOfWordsData,
                               long instanceId,
                               BufferedWriter wData,
                               int row)
                        throws IOException

Throws:

IOException

exportSparseMatrix

protected void exportSparseMatrix(String filename,
                                  SparseData sparseData)
                           throws IOException

export sparse matrix data for use in matlab/R. creates _data.txt with following columns:

• row (int)
• column (int)
• cell value (double)

also exports instance data (instance.txt). By default tab delimited without header. This can be read as a normal 3-column matrix into matlab/R, and then converted into a sparse matrix using Matrix::sparseMatrix (R) or sparse (matlab).

Throws:

IOException

initializeExport

public void initializeExport(InstanceData instanceLabel,
                             Properties properties,
                             SparseData sparseData)
                      throws IOException

get needed properties out of outdir. convert class names into integers for libsvm. attempt to parse the class name into an integer. if this fails, use an index that we increment. index corresponds to class name's alphabetical order.

Specified by:

initializeExport in interface SparseDataFormatter

Throws:

IOException

updateLabelClassMapTransductive

protected void updateLabelClassMapTransductive()

add the 'unlabeled' class id to the classIndexMap if it isn't there already