API Reference

VHash

class vhash.VHash

Vectorizing hash table for fast quantization of text documents

Parameters

largest_ngram: int, optional, default=3

Maximum number of words to take as a single phrase. This table’s vocabulary will consist of phrases from smallest_ngram-words long to largest_ngram-words long.

min_phrase_occurrence: float, default=1E-3

Only the most common phrases are kept. Uncommon phrases are removed if the total count of that phrase across all training documents meets the following criteria:

if min_phrase_occurrence < 1:
    min_count = min_phrase_occurrence * docs.size()
else:
    min_count = min_phrase_occurrence
remove from table if count < min_count

num_features: int, optional, default=1E3

number of features to compute (which equals the dimension of each output dense vector). If fewer training documents than this are used when fitting, then num_features will be reduced to the number of training documents.

max_num_phrases: int, optional, default=1E6

maximum size of term vocabulary, including phrases of all lengths

downsample_to: int, optional, default=100E3

maximum number of documents to use when fitting. If more are provided, then documents are downsampled

live_evaluation_step: int, optional, default=10E3

when fitting a table, to speed things up, infrequent terms are removed every live_evaluation_step number of documents

smallest_ngram: int, optional, default=1

Minimum number of words to take as a single phrase. This table’s vocabulary will consist of phrases from smallest_ngram-words long to largest_ngram-words long.

Methods

`fit`(docs, labels)	Fit model
`fit_transform`(docs, labels)	Fit model, get numeric representation of docs
`transform`(docs)	Get numeric representation of docs

fit(docs: list[str], labels: list) → vhash.vhash.VHash

Fit model

Parameters

docs: list[str]: documents to use to train model
labels: list: class label for each document

Returns

VHash: Calling instance

fit_transform(docs: list[str], labels: list) → nptyping.types._ndarray.NDArray[Any, Any, nptyping.types._number.Float]

Fit model, get numeric representation of docs

Parameters

docs: list[str]: documents to use to train model
labels: list: class label for each document

Returns

numeric: NDArray([Any, Any], float): Numeric representation of documents. rep[x] is for docs[x]. rep[x].size() == num_features (set in constructor)

transform(docs: list[str]) → nptyping.types._ndarray.NDArray[Any, Any, nptyping.types._number.Float]

Get numeric representation of docs

Parameters

docs: list[str]: documents to numerically represent

Returns

numeric: NDArray([Any, Any], float): Numeric representation of documents. rep[x] is for docs[x]. rep[x].size() == num_features (set in constructor)