Skip to main content

module util::Sampling

rascal-0.34.0

Utilities to randomly select smaller datasets from larger datasets

Usage

import util::Sampling;

Dependencies

import util::Math;
import Map;
import List;
import Set;

Description

Sampling is important when the analysis algorithms do not scale to the size of the original corpus, or when you need to train an analysis on a representative set without overfitting on the entire corpus. These sampling functions all assume that a uniformly random selection is required.

function sample

Reduce the arity of a set by selecting a uniformly distributed sample.

set[&T] sample(set[&T] corpus, int target)

A uniform subset is computed by iterating over the set and skipping every element with a probability of 1/(size(corpus) / target). This rapidly generates a new set of expected target size, but most probably a little smaller or larger.

Examples

rascal>import util::Sampling;
ok
rascal>sample({"a","b","c","e","f","g","h","i","j","k"}, 4)
set[str]: {"b","c","e","f","g","h","k"}
rascal>sample({"a","b","c","e","f","g","h","i","j","k"}, 4)
set[str]: {"a","b","c","f","i","j","k"}
rascal>sample({"a","b","c","e","f","g","h","i","j","k"}, 4)
set[str]: {"b","e","h","i","j","k"}

function sample

Reduce the length of a list by selecting a uniformly distributed sample.

list[&T] sample(list[&T] corpus, int target)

The random selection of elements does not change their initial order in the list. A uniform sublist is computed by iterating over the list and skipping every element with a probability of 1/(size(corpus) / target). This rapidly generates a new list of expected target size, but most probably a little smaller or larger.

Examples

rascal>import util::Sampling;
ok
rascal>sample([1..1000], 30)
list[int]: [30,42,133,136,242,245,304,513,538,587,611,725,928,986]
rascal>sample([1..1000], 30)
list[int]: [42,49,51,58,82,134,160,181,240,306,312,318,422,500,507,536,581,645,658,711,719,754,812,841,864,869,953,963,981,997,999]
rascal>sample([1..1000], 30)
list[int]: [10,23,69,127,128,174,244,261,274,418,430,453,500,509,543,561,580,584,634,644,701,730,740,789,812,823,842,894,911,912]

function sample

Reduce the size of a map by selecting a uniformly distributed sample.

map[&T,&U] sample(map[&T,&U] corpus, int target)

A uniform submap is computed by iterating over the map's keys and skipping every key with a probability of 1/(size(corpus) / target). This rapidly generates a new map of expected target size, but most probably a little smaller or larger.