Multilingual Spoken Words Corpus (MSWC)

Provider
Mark Mazumder, Sharad Chitlangia, Colby Banbury et al. (MLCommons, Harvard)
License
CC-BY-4.0 (source)
Size
1,333 files
Stored at
Hugging Face (https://huggingface.co/datasets/MLCommons/ml_spoken_words)
Access
Free and public. No account with the data provider is needed.
Category
Speech & Audio
Source
https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/fe131d7f5a6b38b23cc967316c13dae2-Paper-round2.pdf

About Multilingual Spoken Words Corpus (MSWC)

The Multilingual Spoken Words Corpus is a large audio dataset of spoken words in 50 languages, built for keyword spotting and spoken term search. It contains more than 340,000 keywords and about 23.4 million one-second spoken examples, over 6,000 hours in total. The examples were generated by applying forced alignment to crowd-sourced sentence-level recordings from Mozilla Common Voice to obtain per-word timings and then extracting each word; all alignments are included. Use cases include on-device wake-word and keyword-spotting models, few-shot keyword spotting in low-resource languages and spoken term detection. This repository stores the audio in two formats, 16 kHz wav and 48 kHz opus, under data/wav and data/opus with one folder per language (50 each), and train/dev/test split files under data/splits (measured 2026-11-14; the Hub reports about 930 GB). Configurations are named '{lang}_{format}', for example datasets.load_dataset('MLCommons/ml_spoken_words', 'tt_wav'). Published by MLCommons under CC BY 4.0 for academic research and commercial applications; cite Mazumder et al., NeurIPS 2021 Datasets and Benchmarks.

How to load Multilingual Spoken Words Corpus (MSWC) in Python

from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="MLCommons/ml_spoken_words", filename="data/opus/ar/dev/audio/0.tar.gz", repo_type="dataset")
print(path)

Data structure

Files

Example file: data/opus/ar/dev/audio/0.tar.gz

Analyze this dataset without downloading it

On Scigantic, Multilingual Spoken Words Corpus (MSWC) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Speech & Audio datasets