> ## Documentation Index
> Fetch the complete documentation index at: https://docs.corti.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech to Text Model Training and Refinement

> Learn about how Corti speech to text models are trained and improved

## Architecture

The Corti speech to text pipeline is based on a combination of model architectures. The main workhorse is an encoder-decoder architecture with byte pair encoding (BPE), similar to Whisper. There is also a connectionist temporal classification (CTC)-based architecture that works in tandem with the encoder model.

The combination of those two modeling paradigms provide the following benefits:

* Reduce potential hallucinations
* Improve speed of predictions
* Configure latency to balance response time and accuracy
* Make it easy to integrate with the system

## Finetune

Model and architecture finetuning happens in two ways:

1. Decoder training: Language modeling/terminology learning
2. Encoder training: Auditory model learning

As a starting point, <Tooltip tip="Click here to check the leaderboard">[open source models](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)</Tooltip> are used as baseline parameters upon which various methodologies are employed to refine the parameters of other models.

## Evaluate

Different data sets are used for model finetuning and training than for validation. Assessment methodologies include, but are not limited to the following:

* Word error rate (WER)
* Character error rate (CER)
* Medical term accuracy rate
* Levenshtein distance

<Tip>
  See some of our supporting research here:

  * [Self-Supervised Speech Representation Learning: A Review](https://ieeexplore.ieee.org/abstract/document/9893562)
  * [Do We Still Need Automatic Speech Recognition for Spoken Language Understanding](https://arxiv.org/abs/2111.14842)
  * [Do End-to-End Speech Recognition Models Care About Context](https://arxiv.org/abs/2102.09928)
  * Learn more at [corti.ai/research](https://corti.ai/research)
</Tip>