Detect Candidate Multiword Expressions

Score candidate collocations (multiword expressions) in a quanteda quanteda::tokens object. This wraps quanteda.textstats::textstat_collocations() and returns an export-ready data.table::data.table that can be fed directly to compound_tokens().

Usage

detect_collocations(x, size = 2:3, min_count = 2L, ...)

Arguments

x

A quanteda::tokens object.

size

Integer vector. Length(s) of collocations to score (e.g. 2:3 for bigrams and trigrams).

min_count

Integer. Minimum candidate frequency to be scored.

...

Additional arguments passed to quanteda.textstats::textstat_collocations().

Value

A data.table::data.table with one row per candidate collocation and columns including collocation, count, length, lambda, and z, sorted by descending association strength (lambda).

See also

compound_tokens(), ngram_tokens()

Examples

corp <- quanteda::corpus(c(
  doc1 = "the annual report covered cash flow and annual report risk",
  doc2 = "cash flow guidance featured in the annual report"
))
toks <- tokenize_corpus(corp)
#> 
#> ── Tokenizing corpus ──
#> 
#> ℹ quanteda::tokens() has been called with default parameters
#> ✔ Corpus successfully tokenized
detect_collocations(toks, size = 2, min_count = 2)
#> 
#> ── Detecting collocations ──
#> 
#> ✔ Found 3 candidate collocations
#>      collocation count count_nested length   lambda        z
#>           <char> <int>        <int>  <num>    <num>    <num>
#> 1: annual report     3            0      2 5.241747 2.510402
#> 2:     cash flow     2            0      2 4.976734 2.354187
#> 3:    the annual     2            0      2 3.806662 2.147971