Languages of the World

When developing open-source models and datasets for low-resource languages like Kinyarwanda, I consistently ran into the same architectural bottleneck: linguistic metadata is scattered everywhere. Whether you need ISO 639-3 codes, Glottolog family trees, or regional speaker counts, the data is typically siloed across mismatched CSVs, XML files, and SPARQL endpoints. Working as computational linguist with many languages, I needed a way to instantly access and cross-reference this global information without building custom data wrangling pipelines for every new NLP project.

That is why I created low (Languages of the World), a lightweight, read-only Python utility that aggregates seven major open linguistic datasets into a single, connected in-memory object graph. Instead of hiding data behind heavy, traditional repository classes or requiring complex database queries, low exposes everything through easy-to-understand Python sequences. Using direct dot-notation, you can effortlessly traverse from a language, to its parent language family, to the countries where it is spoken, and finally to its regional speaker counts. Every entity type features smart multi-key .get() and .filter() lookups, making data extraction highly intuitive.

What’s Inside?

Under the hood, low pulls together, normalizes, and deduplicates data from SIL ISO 639-3, UN M49, Google’s LinguaMeta, Glottolog, Unicode CLDR, the CIA World Factbook, and Wikidata. The resulting graph provides a massive, queryable snapshot of global linguistics, featuring:

• ~7,900 languages - ISO 639-3 codes, labels, scope (individual / macrolanguage), and optional ISO 639-1 codes
• Country assignments - languages linked to the countries where they are spoken
• 247 countries - ISO 3166-1 alpha-2 codes, population, and back-references to spoken languages
• 5 continents & 17 UN M49 regions - geographic hierarchy from country up to continent
• ~4,800 Glottolog family nodes - navigable parent/child tree with 246 root families
• Endangerment status - Glottolog Agglomerated Endangerment Scale (AES) per language
• 106 writing systems - ISO 15924 scripts with primary-script assignment per language
• Cross-lingual names - canonical endonyms and exonyms across languages
• Per-country speaker counts - from CLDR, CIA World Factbook, LinguaMeta, and optional web-scraped data
• Global speaker totals - merged across LinguaMeta and Wikidata
• Official language status - nationally official, regionally official, and de facto official languages per country

Links

• github.com/jnehring/languages-of-the-world
• pypi.org/project/languages-of-the-world