lipimala — Indic Script Converter

lipimala is a deterministic Indic transliteration suite focused on exact Unicode source preservation, canonical script rendering, Vedic-aware conversion, and synchronized behavior across Dart, JavaScript/Node.js, Python, and PHP.

It supports Latin/IAST-style input, Devanagari, Gujarati, canonical reverse conversion, direct Devanagari ↔ Gujarati conversion, and Plain-English/Hunterian transcription.

Design goal: do not confuse a canonical reverse transliteration with the exact original source.
lipimala keeps those concepts separate and can recover the original source through a structured result envelope or checksummed Unicode-Tag metadata.

---

Runtimes & registries

Runtime	Package directory	Registry	Install	Primary import
Dart 3.4+	dart/	pub.dev	dart pub add lipimala	import 'package:lipimala/lipimala.dart';
Node.js 20+ / ES2023+	javascript/	npm	npm install lipimala	import { toDevanagari } from 'lipimala';
Python 3.12+	python/	PyPI	pip install lipimala	from lipimala import to_devanagari
PHP 8.3+	php/ (submodule)	Packagist	composer require jayeshmepani/lipimala	use function Lipimala\\toDevanagari;

The repository is a feature-parity monorepo. Each runtime has its own native implementation, package manifest, tests, examples, and quality tooling.

PHP source of truth: jayeshmepani/indic-script-converter-php.
The monorepo php/ path is a git submodule that links to that repository (clicking it on GitHub opens the PHP-only repo).
Packagist versions/tags are managed on the PHP repository, not via monorepo subtree sync.

---

Why lipimala?

Most transliteration APIs answer one question:

source text → rendered text

lipimala also models the distinction between:

canonical rendered representation
            ≠
exact original source identity

That matters whenever multiple source spellings or Unicode representations collapse to the same visible Brahmic text.

For example, a Devanagari rendering cannot naturally preserve Latin casing, source alias choice, or whether the caller supplied NFC or NFD. lipimala can retain those distinctions separately and restore the original source later.

Two exact-recovery strategies

1. TransliterationResult envelope
   Keeps the exact source, normalized input, rendered form, profile, normalization settings, diagnostics, and integrity information together.

2. Checksummed Unicode-Tag trailer
   embedExactSourceMetadata: true appends an invisible LIT1: metadata payload to the rendered string so exact recovery can travel with that string.

The metadata trailer is intended for controlled Unicode-preserving storage/transport. External systems may strip or sanitize Unicode Tag characters. For durable interchange, the structured result/JSON envelope is the safer choice.

---

Core capabilities

Metadata-backed exact source round-trip

lipimala can preserve and recover source-string distinctions that the visible destination script cannot represent, including:

• original case
• NFC vs NFD representation
• precomposed vs decomposed sequences
• accepted alias choice
• combining-mark ordering
• punctuation
• whitespace
• supplementary Unicode characters

This is exact source recovery, not a claim that every visible transliteration mapping is inherently bijective.

Canonical, smart, and exact reverse APIs

Reverse conversion deliberately separates three behaviors:

• Canonical — derive canonical Latin from the visible Brahmic text.
• Smart — recover embedded exact source when valid metadata is present; otherwise use canonical reverse conversion.
• Exact — require valid exact-source metadata and fail when it is unavailable or invalid.

This distinction is available for Brahmic → IAST and for direct Devanagari ↔ Gujarati conversion.

Direct Devanagari ↔ Gujarati conversion

The direct converter does not need to pivot through IAST.

It handles:

• direct script mapping
• nukta combinations
• script digits
• whitespace policy
• unknown-character policy
• Unicode normalization
• Vedic marks
• metadata-backed exact source recovery

Where the visible Gujarati and Devanagari repertoires are many-to-one, canonical conversion remains canonical while exact source identity is preserved separately when requested.

Vedic-aware Unicode handling

Vedic marks are treated as a first-class requirement rather than discarded during ordinary conversion.

The suite preserves encoded Vedic accent marks and maintains the intended Unicode storage ordering around vowels/mātrās, bindu/visarga, and svara marks.

Plain English / Hunterian transcription

Plain-English and Hunterian views are intentionally human-readable and may be intrinsically lossy as visible representations.

The result-envelope architecture allows the display form to be lossy while the original source can still remain recoverable.

Four-runtime parity

Dart, JavaScript, Python, and PHP are maintained against the same behavior and shared verification material.

The current shared verification set contains:

• 497 transliteration cases
• 22 Vedic fixtures

Parity is verified against the shared corpus/golden outputs rather than inferred merely because the APIs have similar names.

---

Supported directions

Source	Target	Modes
Latin / IAST / extended Indic	Devanagari	string, envelope, metadata-backed exact
Latin / IAST / extended Indic	Gujarati	string, envelope, metadata-backed exact
Latin / IAST	Plain English / Hunterian	string, envelope
Devanagari	IAST	canonical, smart, exact
Gujarati	IAST	canonical, smart, exact
Devanagari	Gujarati	canonical, smart, exact
Gujarati	Devanagari	canonical, smart, exact

---

Romanization profiles

Forward Latin → Brahmic conversion supports profile-scoped behavior.

strictIast

Classical Sanskrit IAST inventory.

Use it when input should be interpreted strictly according to the supported IAST rules.

iso15919Core

The explicitly implemented ISO-15919-style core profile.

Use it when you need the supported ISO-style distinctions beyond strict Sanskrit IAST.

extendedIndic

The practical default profile.

It extends the accepted input inventory with regional/modern Indic and compatibility aliases, including supported nukta-oriented forms.

The profile names describe the exact behavior implemented by lipimala. iso15919Core should not be interpreted as a claim to implement every possible ISO 15919 feature outside the documented table.

---

Unicode normalization

The APIs expose explicit normalization control:

preserve
nfc
nfd

Typical envelope defaults are:

inputNormalization  = nfd
outputNormalization = nfc

Normalization used for parsing/rendering is kept conceptually separate from the exact original source stored by the lossless envelope/metadata path.

---

Forward conversion options

The Devanagari and Gujarati forward converters expose corresponding option sets.

Key options include:

Option	Purpose
profile	strictIast, iso15919Core, or extendedIndic
unknownLatinPolicy	preserve/pass through, bracket, or throw
digitPolicy	preserve ASCII digits or convert to script digits
punctuationPolicy	preserve punctuation or use Indic danda behavior
omPolicy	transliterate letters or use the script OM sign
ambiguousLPolicy	resolve ambiguous ḷ behavior
acceptAsciiLongVowels	accept ASCII long-vowel aliases
acceptPlainSh	accept plain sh compatibility input
acceptPlainXAsKha	compatibility handling for x
acceptWAsVa	compatibility handling for w
preserveVedicAccentMarks	preserve supported Vedic accents
collapseWhitespace	collapse whitespace runs
embedExactSourceMetadata	append exact-source Unicode-Tag metadata

---

Direct-script options

IndicScriptConversionOptions controls Devanagari ↔ Gujarati conversion.

Option	Values / behavior
inputNormalization	preserve, nfc, nfd
outputNormalization	preserve, nfc, nfd
unknownPolicy	preserve or throw
digitPolicy	convert to target digits or preserve source digits
collapseWhitespace	collapse contiguous whitespace
embedExactSourceMetadata	append typed exact-source metadata

---

Quick start

Dart

import 'package:lipimala/lipimala.dart';

void main() {
  final result = 'Kṛṣṇa ā́tman'.toDevanagari();

  print(result.rendered);          // कृष्ण आ॑त्मन्
  print(result.restoreOriginal()); // Kṛṣṇa ā́tman
}

JavaScript / Node.js

import { toDevanagari } from 'lipimala';

const result = toDevanagari('Kṛṣṇa ā́tman');

console.log(result.rendered);          // कृष्ण आ॑त्मन्
console.log(result.restoreOriginal()); // Kṛṣṇa ā́tman

Python

from lipimala import to_devanagari

result = to_devanagari('Kṛṣṇa ā́tman')

print(result.rendered)           # कृष्ण आ॑त्मन्
print(result.restore_original()) # Kṛṣṇa ā́tman

PHP

<?php

declare(strict_types=1);

use function Lipimala\toDevanagari;

$result = toDevanagari('Kṛṣṇa ā́tman');

echo $result->rendered, PHP_EOL;          // कृष्ण आ॑त्मन्
echo $result->restoreOriginal(), PHP_EOL; // Kṛṣṇa ā́tman

---

Exact-source string workflow

When you need a single rendered string that can later recover its exact source, enable embedded metadata.

Conceptually:

Kṛṣṇa
   ↓
कृष्ण + invisible checksummed Unicode-Tag metadata
   ↓
Kṛṣṇa

The visible text remains ordinary script text; the exact-source payload is non-rendering metadata.

Use the corresponding exact reverse API when exact provenance is mandatory, or the smart API when canonical fallback is acceptable.

---

TransliterationResult

The structured envelope carries conversion state explicitly.

Its cross-runtime model includes fields/concepts equivalent to:

original
normalizedInput
rendered
profile
inputNormalization
outputNormalization
renderingIsInjective
issues
originalCodePoints

and operations equivalent to:

restoreOriginal()
hasErrors
toJson()
fromJson()
toJsonText()
fromJsonText()

The envelope makes it possible for a rendered view to be intentionally non-injective while the original source remains independently recoverable.

---

Public API examples & technical documentation

Comprehensive examples exercise the public APIs and option permutations.

Runtime	Example	Run
Dart	dart/example/public_api_examples.dart	cd dart && dart run example/public_api_examples.dart
JavaScript	javascript/examples/public-api-examples.js	node javascript/examples/public-api-examples.js
Python	python/examples/public_api_examples.py	PYTHONPATH=python python3 python/examples/public_api_examples.py
PHP	php/examples/public_api_examples.php	php php/examples/public_api_examples.php

For the full API surface, reverse modes, metadata helpers, result structures, enums, and runtime-specific naming, see the project's API reference/documentation.

---

Monorepo structure

lipimala/
├── dart/
├── javascript/
├── python/
├── php/                 # git submodule → indic-script-converter-php
├── shared/
│   └── verification-output/
├── .gitmodules
├── README.md
├── CHANGELOG.md
└── LICENSE

Each runtime directory is independently packageable and contains its own implementation, examples, tests, and quality configuration.

Clone with the PHP submodule

git clone --recurse-submodules https://github.com/jayeshmepani/indic-script-converter.git
# or after a normal clone:
git submodule update --init --recursive

Keep the PHP submodule pin up to date

The monorepo stores a fixed commit for php/ (not a live symlink). Refresh options:

1. Automatic (recommended)

   • Push to indic-script-converter-php main → GitHub Action notifies the monorepo → monorepo Action updates the pin and commits.
   • Requires secret MONOREPO_SYNC_TOKEN on the PHP repo (PAT that can dispatch workflows on the monorepo).
   • Backup: monorepo also syncs on a 6-hour schedule and via Actions → Sync PHP submodule → Run workflow.

2. Manual

cd php && git pull origin main && cd ..
git add php
git commit -m "chore: bump php submodule"
git push origin main

Packagist releases still use tags on the PHP repository only; submodule sync does not publish packages.

Development & parity testing

Each implementation has its own quality command:

Dart        cd dart       && make quality
JavaScript  cd javascript && npm run quality
Python      cd python     && make quality
PHP         cd php        && composer quality   # requires submodule checked out

The runtime suites validate behavior against the shared verification corpus and golden outputs.

When changing conversion behavior, update and verify all four runtimes so that a conversion performed by one implementation remains behaviorally aligned with the others. Prefer editing PHP sources in the standalone PHP repository, then bump the submodule here.

---

Project scope

lipimala deliberately prioritizes fidelity over script count.

It is designed for applications that care about:

• deterministic transliteration
• scholarly/structured Latin input
• exact source provenance
• Unicode normalization behavior
• Gujarati and Devanagari fidelity
• Vedic text handling
• cross-runtime reproducibility
• archival or round-trip workflows

It is not intended to compete with broad transliteration engines on total script or romanization-scheme count.

---

Exactness terminology

In this project:

• Canonical round-trip means deriving the canonical source representation from visible script text.
• Exact source round-trip means recovering the original source string through preserved provenance.
• Lossless envelope means the structured result retains enough information to restore the original source independently of whether the rendered target is injective.
• Integrity checked means metadata/envelope consistency is validated before exact recovery; it does not mean cryptographic authentication.

---

License

This project is licensed under the MIT License.