Skip to main content

​
DataPreparationConfig

​
input_path
str | readable object
required
Source location for raw data files or a readable object from which data can be accessed.
​
output_path
str | writable object
required
Destination location for processed data files or a writable object where processed data will be stored.
​
tokenization
str | TimeSeriesTokenizerConfig | callable
required
Tokenization strategy based on data modality:
  • For text/string/code: String denoting a Hugging Face tokenizer name or path to a local tokenizer compatible with Hugging Face
  • For time series: An instance of TimeSeriesTokenizerConfig
  • For custom tokenization: A callable function that can map string → tensor of floats/integers (2D for patch-based)
​
max_sequence_length
int
default:"4096"
Maximum sequence length for tokenized data.

​
TimeSeriesTokenizerConfig

​
type
str
required
Tokenization approach - "patch_based" (Chronos Bolt, TSFM, TiRex style) or "bin_quant_based" (Chronos style).
​
input_patch_size
int | None
default:"None"
Size of input patches. Required for patch-based tokenization. Default: None for bin quantization
​
output_patch_size
int | None
default:"None"
Size of output patches. Required for patch-based tokenization. Can differ from input patch size (e.g., TimesFM uses longer output patches than input). Default: None for bin quantization
​
patch_masking
bool | None
default:"False"
Enables patch masking strategy. Only applicable for patch-based tokenization. Helps models learn to predict well for context lengths that are multiples of input patch length (see TimesFM paper).
​
number_of_bins
int | None
default:"4096"
Number of quantization bins. Required for bin quantization-based tokenization.
​
normalization_method
str | callable
default:"z-norm"
Normalization strategy. Accepts custom function that maps a list of numbers to a list of numbers.
​
normalization_mean
float | None
default:"None"
Mean value for normalization. Should remain None for custom normalization methods. For z-norm, computes mean over first patch (patch-based tokenization, following TimesFM) or entire series (bin quantization, following Chronos).
​
normalization_std
float | None
default:"None"
Standard deviation for normalization. Computes series-wise standard deviation based on first patch (patch-based) or entire series (bin quantization).
The system automatically handles padding, missing values, and end-of-sequence tokens.