CVSS: Multilingual Speech-to-Speech Translation Corpus, 21 Languages into English (CVSS-C and CVSS-T, Google Research)
- Provider
- Google Research (Ye Jia, Michelle Tadmor Ramanovich, Quan Wang, Heiga Zen)
- License
- CC-BY-4.0 (source)
- Stored at
- Google Cloud Storage, gs://cvss (us)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Speech & Audio
- Source
- https://arxiv.org/abs/2201.03713
About this dataset
CVSS is a sentence-level speech-to-speech translation (S2ST) corpus pairing speech in 21 source languages with spoken English translations. It is derived from the Mozilla Common Voice speech corpus (release version 4) and the CoVoST 2 speech-to-text translation corpus: the English translation text of CoVoST 2 was synthesized into speech by text-to-speech. Two versions are provided. CVSS-C (cvss_c_v1.0/) has all target speech in a single canonical speaker's voice; CVSS-T (cvss_t_v1.0/) transfers the voice of each source speaker to the translation, for voice-preserving translation. Each version has about 1,900 hours of speech together with the Common Voice source audio. Source languages: Arabic, Catalan, Welsh, German, Estonian, Spanish, Persian, French, Indonesian, Italian, Japanese, Latvian, Mongolian, Dutch, Portuguese, Russian, Slovenian, Swedish, Tamil, Turkish and Chinese. Each per-language tar.gz (for example cvss_c_de_en_v1.0.tar.gz) holds train, dev and test folders of translation audio clips and train/dev/test.tsv files with normalized translation text (numbers, currencies and acronyms written as spoken). Clips are paired with Common Voice v4 source audio by file name. Used to train and evaluate direct speech translation models such as Translatotron. License: CC BY 4.0. Cite Jia et al., CVSS Corpus and Massively Multilingual Speech-to-Speech Translation, LREC 2022 (arXiv:2201.03713).
How to access this dataset in Python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('cvss/', detail=False)[:10]Lists the files with anonymous access. Runs on any machine, with no cloud account.
Data structure
Files
Example file: cvss_c_v1.0/cvss_c_ar_en_v1.0.tar.gz
Analyze this dataset without downloading it
On Scigantic, CVSS: Multilingual Speech-to-Speech Translation Corpus, 21 Languages into English (CVSS-C and CVSS-T, Google Research) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Speech & Audio datasets
- Multilingual Librispeech
- E-GMD: Expanded Groove MIDI Dataset, 444 Hours of Human Drumming on 43 Drum Kits with Aligned MIDI and Audio (Google Magenta)
- CocoChorales: 240,000 Synthesized Four-Part Chorale Mixtures with Per-Instrument Stems, MIDI and Note Expression (Google Magenta)
- MInDS-14: multilingual spoken intent detection
- Nemotron Content Safety Audio Dataset
- Voxpopuli
- Groove MIDI Dataset (GMD): 13.6 Hours of Expressive Human Drumming as MIDI with Synthesized Audio (Google Magenta)
- Earnings-21: entity-dense earnings call ASR benchmark