Michał Kornacki, PhD

1. Introduction & Design Philosophy

This document describes a working Translation Management System (TMS) developed for the Translation Technology course at the University of Łódź. It is not a theoretical blueprint. Every component described here has been implemented, tested with live jobs, and refined through three design increments. Students can read this document and build a comparable system from scratch.

That said, the architecture does not belong to a single institution. The same SQLite database, the same folder tree, and the same agent pipeline can serve a solo freelancer, a small agency, or an NGO running multi-year programmes across three continents.

Who This Document Is For

Four groups of readers will get the most from this document.

Translation Studies students. If you are studying at a university Translation Studies programme, this document is a practical manual for building translation infrastructure. You will learn how to structure a database, how to route jobs through a state machine, how to enforce quality gates, and how to build a reusable corpus. Every section includes enough detail that you can reimplement the system on a standard Linux workstation without guessing.

Solo freelance translators. A single practitioner running their own practice needs three things: a private translation memory that grows with every job, a self-audit mechanism that catches errors before the client sees them, and an audit log that defends against disputes. The TMS provides all three. No adaptation is required — the solo mode is the default.

Freelancers who want a custom system. The owner built this TMS not to sell a product but to demonstrate that a single freelancer can own their entire pipeline. Every script is plain Python: if your clients prefer British English instead of American, change one variable in the requirements template. If you need a new file format, write a twenty-line handler and add it to the format CHECK constraint. If you want stricter MQM thresholds for sworn translations and looser ones for internal emails, edit the severity weights in the requirements-gathering questionnaire. The system is not a black box — it is a starter kit that assumes you will modify it. This document shows you the default configuration; your configuration will differ, and that is the point.

NOTE! This is version 1.2 – it is not final and requires further tailoring.

What Problem Does It Solve?

Translators everywhere handle heterogeneous document streams: conference abstracts, medical consent forms, lease agreements, software strings, book chapters, sworn diplomas. Commercial CAT tools — SDL Trados, MemoQ — assume a professional agency workflow with project managers, vendor pools, and invoice pipelines. That assumption does not map neatly onto a solo translator, a university course, or a non-profit with one staff linguist. Spreadsheets degrade once you have more than a few dozen jobs. Email folders lose track of versions. Shared drives overwrite each other's files.

The TMS described here fills that gap: a lightweight, agent-assisted system that tracks state, enforces quality checks, and builds a reusable corpus without the overhead of enterprise licensing. It is small enough for one person and structured enough for a team of ten.

Why AI Agents Rather Than Traditional CAT Tools?

Traditional CAT tools excel at segment-level translation memory, terminology matching, and format preservation. They do not, however, reason about register, audience, or institutional context. An AI agent can read a source text alongside a set of context documents and ask: “This is a sworn translation for a legalised diploma — should I use British or American conventions?” A CAT tool cannot ask that question. The trade-off is that the agent needs structure: a database to record state, a file system to isolate versions, and a quality framework to stop bad translations from reaching the client. This document provides that structure.

A Different Paradigm: Context First, Segments Last

This document does not describe a lightweight CAT tool with fewer features. It describes a fundamentally different approach to translation memory. Commercial CAT tools begin with segmentation: they chop a document into sentences, match each sentence against a database of previous sentences, and suggest replacements. This architecture excels when source texts repeat verbatim, but it struggles with morphologically rich languages such as Polish, where inflection means that even slight grammatical variation defeats exact-match retrieval and where fuzzy matching returns noisy, low-value suggestions.

The TMS reverses the priority. It begins with the document as a whole. Before translating, the Translator queries QMD using the first 500 words of the source text, retrieving conceptually similar whole documents from the corpus. The translator reads those documents for register, terminology, and style cues — not isolated sentences, but paragraphs in context. Only as a last resort does the system fall back to the segment_cache for verbatim phrase reuse. The segment cache is intentionally primitive: it is a lightweight fallback, not the primary memory mechanism.

This is not a replacement for SDL Trados or MemoQ. It is an alternative architecture for translators whose work is too heterogeneous, too context-dependent, or too morphologically complex for segment-first matching to add value.

Design Philosophy: KISS and Gall's Law

The system was built according to two principles.

KISS (Keep It Simple, Stupid): every feature added after the prototype was justified by a concrete failure or bottleneck. There is no plugin architecture, no REST API, no container orchestration. The entire system is a SQLite file, a folder tree, a handful of Python scripts, and an HTML viewer.

Gall's Law: “A complex system that works is invariably found to have evolved from a simple system that worked.” The TMS began as three database tables and four job states. Corpus storage, AI review, and QMD semantic search were added only after the basic pipeline proved it could ingest, translate, and deliver a document without losing data.

Organisational Contexts

The architecture was born in a university Translation Studies programme. It is not trapped there. Below are three contexts where the same components apply, with notes on what each context needs and what (if anything) must change. Another five are listed as possible implementations after heavy modifications (methodology and practical application).

MAIN:

1. Solo freelance translators. A single translator running their own practice needs three things: a private translation memory that grows with every job, a self-audit mechanism that catches errors before the client sees them, and an audit log that defends against disputes. The TMS provides all three. No adaptation is required — the solo mode is the default.

2. Small translation agencies (2–10 translators). Shared terminology becomes essential once more than one person translates for the same client. The SQLite terminology table is readable by every translator; approved entries propagate instantly. The AI reviewer enforces the same QA standard across the team, and client-facing MQM reports justify revision requests or price premiums. The only change needed is a convention: one designated “Orchestrator” operator who handles database writes, since SQLite does not support concurrent writers.

3. NGOs and non-profits. Donor-funded programmes often require audit trails: who translated what, when, and at what quality level. The audit_log table answers that question without extra paperwork. Terminology consistency across multi-year projects prevents “sustainable development” from becoming three different Polish phrases in consecutive annual reports. The system carries zero licensing cost, which matters when the budget is grants and donations.

OPTIONAL:

1. Legal practices and notary offices. Certified translations demand format fidelity and translator accountability. The placeholder protection layer preserves case numbers, statute references, and party names. The AI reviewer flags Critical errors — mistranslated damages figures or misidentified parties — before the document carries a translator's stamp. The audit log provides the traceability that sworn-translator regulations increasingly require. No schema changes are needed, though the requirements-gathering phase should explicitly ask whether the output needs a sworn-translator clause.

2. Publishing houses (book translation). Long-form texts break most lightweight systems. The TMS handles them through chunked AI review: a 60,000-word novel is reviewed in 2,000-word slices, each scored separately. Corpus search lets a translator check whether the protagonist's voice was consistent across chapters translated six months apart. The segment cache stores recurring phrases — “He drained his glass” — so the translator does not reinvent the wheel on page 400.

3. Healthcare and medical translation. Patient safety turns translation quality into a liability issue. The AI reviewer treats Critical errors as a hard gate: a single mistranslated drug dosage or anatomical reference halts the pipeline until it is fixed. The terminology bank stores drug names, procedure names, and anatomical terms with domain tags, so “appendix” is never confused between vermiform and document. Adaptation: the requirements phase should flag medical jobs explicitly so the AI reviewer loads the medical severity rubric.

4. Software localisation (post-editing). User-interface strings repeat across products and versions. The segment cache serves as a lightweight TM for exact-match reuse; placeholder protection ensures that variables like %s and {username} survive translation untouched. The MTPE pipeline treats machine-translated source as a draft file type, triggering post-editing rather than translation from scratch. The only adaptation is format: source files are often .po, .xliff, or .json, which are not in the default format list but can be added by extending the format CHECK constraint and writing a word-count handler.

5. Subtitling and audiovisual translation. Recurring dialogue — “Cut!” — benefits from segment cache lookup. Timing constraints are not enforced by the TMS itself, but the AI reviewer can be configured to flag length violations (a target line exceeding 42 characters) as a Locale Conventions or Design error. Adaptation: source files are typically .srt or .vtt; like software strings, these require a format handler extension.

Domain Adaptability

The same architecture handles different text domains because the requirements phase, the terminology layer, and the MQM framework are all domain-agnostic. What changes is the emphasis.

The principle is simple: the pipeline does not care whether the text is a poem or a patent. The agents care, because the requirements phase tells them which rules to load. The database stores the domain tag in corpus_entries.domain_tag so that future queries can filter by field.

Contexts Where This System Would Not Prove Its Worth

An honest architecture document admits its own boundaries.

Enterprise LSPs (50+ translators). SQLite's file-level locking and single-writer design cannot scale to a large agency with simultaneous project managers, translators, and proofreaders all writing to the same database. An enterprise LSP needs PostgreSQL, a REST API, and role-based access control. The TMS is not that.

Real-time interpretation. The TMS is asynchronous by design: files drop, agents process, states advance. Simultaneous interpretation happens in seconds, not hours. The architecture is the wrong shape entirely.

High-volume MT-only pipelines. If the goal is to translate ten million words per day for the lowest possible cost, raw machine translation with no human review is cheaper. The TMS incurs agent-compute and human-proofreading cost that only makes sense when quality, not throughput, is the priority.

Non-text translation. Image OCR post-editing, audio transcription, and video dubbing are out of scope. The pipeline assumes a text file entering at one end and a text file leaving at the other.

Scope

• Language pair: Polish ↔ English (default). The language_pair column enforces this with a CHECK constraint. Extending the architecture to other pairs requires modifying that constraint and adapting the agent profiles (terminology expertise, locale conventions, MQM error patterns). The schema itself is pair-agnostic.
• Direction: Source-to-target or target-to-source (for back-translation or verification jobs).
• Domain: Adaptable across academic, administrative, certified, general, legal, literary, marketing, medical, and technical texts. The requirements-gathering phase selects the appropriate register, terminology set, and MQM severity weights.
• Format support: .txt, .md, .html, .odt, .pdf, .docx, .xlsx, .ods, .csv, .rtf. Additional formats (.po, .xliff, .srt, .vtt) can be added by extending the format CHECK constraint and writing a word-count handler.

2. System Architecture Overview

High-Level Architecture

The TMS is organised around a single source of truth — the SQLite database — with five functional layers surrounding it:

+---------------------+
|   Owner / Human     |
|   (proofreading,     |
|    sign-off, inbox) |
+----------+----------+
           |
+----------v----------+     +------------------+
|   Orchestrator      |<--->|   HTML Viewer    |
|   (routes tasks,    |     |   (projects,     |
|    manages state)   |     |    filters,      |
|                     |     |    file links)   |
+----------+----------+     +------------------+
           |
    +------+------+
    |             |
    v             v
+------+       +--------+
|Translator    |  AI    |
|Agent         |Reviewer |
|(Polish-EN)   | Agent   |
+------+       +--------+
    |             |
    +------+------+
           |
+----------v----------+
|  File Handler /     |
|  Cataloguer         |
|  (word counts, DB   |
|   writes, moves)    |
+----------+----------+
           |
+----------v----------+
|  Ingest / Repo      |
|  Manager            |
|  (corpus, terminology|
|   QMD updates)      |
+----------+----------+
           |
+----------v----------+
|  SQLite Database    |
|  (single source of  |
|   truth)            |
+---------------------+
           |
+----------v----------+
|  File System        |
|  (project folders,  |
|   corpus, inbox)    |
+---------------------+
    

Core Components

Data Flow

A job enters the system through the inbox and progresses through phases:

Inbox drop → Project creation → Requirements → Translation → AI Review → (Fixes) → Proofreading → Final Target → Delivery → Corpus & Terminology Ingest → Close

Two-Layer State Machine

The system uses two columns to track progress: a coarse status (4 states) and a fine-grained sub_status (9 phases). Only the Orchestrator advances status; agents advance sub_status within the boundaries of the current status.

The rule is simple: status controls what can happen next. sub_status records where you are. Only the Orchestrator advances status; agents advance sub_status within the boundaries of the current status.

3. Database Architecture

Why SQLite?

The system could have used a spreadsheet. It does not, for reasons that become obvious once you have handled more than twenty jobs:

Full Schema (7 Tables)

The canonical database lives at Translations/translations.db. All tables use INTEGER PRIMARY KEY AUTOINCREMENT for surrogate IDs. Foreign keys are enforced.

clients

CREATE TABLE clients (
    client_id INTEGER PRIMARY KEY AUTOINCREMENT,
    client_name TEXT NOT NULL UNIQUE,
    contact_info TEXT,
    notes TEXT,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_clients_name ON clients(client_name);

projects

CREATE TABLE projects (
    project_id INTEGER PRIMARY KEY AUTOINCREMENT,
    project_key TEXT NOT NULL UNIQUE,            -- e.g. ID2026-01-01-001
    client_id INTEGER NOT NULL REFERENCES clients(client_id),
    language_pair TEXT NOT NULL CHECK(language_pair IN ('EN-PL','PL-EN')),
    direction TEXT NOT NULL CHECK(direction IN ('source_to_target','target_to_source')),
    start_date TEXT NOT NULL DEFAULT (datetime('now')),
    delivery_date TEXT,
    status TEXT NOT NULL DEFAULT 'PENDING'
        CHECK(status IN ('PENDING','ACTIVE','REVIEW','DONE')),
    sub_status TEXT DEFAULT 'IDENTIFYING'
        CHECK(sub_status IN (
            'IDENTIFYING','REQUIREMENTS','TRANSLATING',
            'MQM_REVIEW','FIXING','PROOFREADING',
            'COMPLETE','REPO_INGEST','CLOSED'
        )),
    summary TEXT,
    translator_agent TEXT,
    reviewer_agent TEXT,
    owner_sign_off TEXT,
    created_at TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at TEXT NOT NULL DEFAULT (datetime('now')),
    word_count INTEGER,
    price REAL,
    currency TEXT NOT NULL DEFAULT 'PLN'
        CHECK(currency IN ('PLN','EUR','USD','GBP')),
    paid_status TEXT DEFAULT 'NOT PAID'
        CHECK(paid_status IN ('PAID','NOT PAID')),
    invoice_pdf_path TEXT,
    invoice_number TEXT,
    domain TEXT CHECK(domain IN ('academic','administrative','certified','general','legal','literary','marketing','medical','technical')),
    job_type TEXT CHECK(job_type IN ('translation','proofreading','other'))
);

Key columns added since v1.0:

• domain: the text domain selected during requirements gathering. Enforced by CHECK constraint.
• job_type: translation, proofreading, or other. Determines which workflow phases are active.
• reviewer_agent: records which AI reviewer was assigned (replaces the earlier critic_agent naming).
• owner_sign_off: timestamp or status string recording explicit owner approval before CLOSE.

project_files

CREATE TABLE project_files (
    file_id INTEGER PRIMARY KEY AUTOINCREMENT,
    project_id INTEGER NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
    file_name TEXT NOT NULL,
    file_path TEXT NOT NULL,
    file_type TEXT NOT NULL
        CHECK(file_type IN ('source','draft','mqm_reviewed','fixed','proofreading','target','context','terminology')),
    format TEXT NOT NULL
        CHECK(format IN ('txt','odt','pdf','html','md','docx','xlsx','ods','csv','rtf','other')),
    checksum TEXT,
    word_count INTEGER,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

Note: word_count is stored per file as well as per project, enabling granular billing and progress tracking.

corpus_entries

CREATE TABLE corpus_entries (
    entry_id INTEGER PRIMARY KEY AUTOINCREMENT,
    project_id INTEGER NOT NULL REFERENCES projects(project_id),
    source_document TEXT NOT NULL,
    target_document TEXT NOT NULL,
    language_pair TEXT NOT NULL,
    domain_tag TEXT,
    source_file_name TEXT,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_corpus_pair ON corpus_entries(language_pair);
CREATE INDEX idx_corpus_domain ON corpus_entries(domain_tag);
CREATE INDEX idx_corpus_project ON corpus_entries(project_id);

terminology_entries

CREATE TABLE terminology_entries (
    term_id INTEGER PRIMARY KEY AUTOINCREMENT,
    source_term TEXT NOT NULL,
    target_term TEXT NOT NULL,
    language_pair TEXT NOT NULL,
    part_of_speech TEXT,
    domain TEXT,
    context_note TEXT,
    source_project_id INTEGER REFERENCES projects(project_id),
    approved BOOLEAN DEFAULT FALSE,
    occurrence_count INTEGER DEFAULT 1,
    created_at TEXT NOT NULL DEFAULT (datetime('now')),
    confidence_score REAL,
    updated_at TEXT,
    source_file TEXT
);

CREATE INDEX idx_term_source ON terminology_entries(source_term);
CREATE INDEX idx_term_target ON terminology_entries(target_term);
CREATE INDEX idx_term_pair ON terminology_entries(language_pair);
CREATE INDEX idx_term_domain ON terminology_entries(domain);
CREATE INDEX idx_term_approved ON terminology_entries(approved);
CREATE INDEX idx_term_dedup ON terminology_entries(source_term COLLATE NOCASE, target_term COLLATE NOCASE, language_pair);

Key columns:

• source_file: provenance tracking (e.g. "general_glossary.xlsx", "medical_terms.txt"). Allows filtering by origin in the viewer.
• approved: BOOLEAN. The schema default is FALSE, but the manual ingest pipeline (ingest_termbase.py) sets approved = TRUE for all curated entries because they are pre-verified by the owner.
• confidence_score: reserved for future automated extraction scoring. Currently NULL for manual ingest.

audit_log

CREATE TABLE audit_log (
    log_id INTEGER PRIMARY KEY AUTOINCREMENT,
    project_id INTEGER REFERENCES projects(project_id),
    agent_name TEXT NOT NULL,
    action TEXT NOT NULL,
    details TEXT,
    timestamp TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_audit_project ON audit_log(project_id);
CREATE INDEX idx_audit_agent ON audit_log(agent_name);
CREATE INDEX idx_audit_timestamp ON audit_log(timestamp);

segment_cache

CREATE TABLE segment_cache (
    segment_id INTEGER PRIMARY KEY AUTOINCREMENT,
    source_segment TEXT NOT NULL,
    target_segment TEXT NOT NULL,
    language_pair TEXT NOT NULL,
    project_id INTEGER REFERENCES projects(project_id),
    domain TEXT,
    quality_score REAL,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_seg_source ON segment_cache(source_segment);
CREATE INDEX idx_seg_pair ON segment_cache(language_pair);

WAL Mode

The database is created with:

PRAGMA journal_mode = WAL;

Write-Ahead Logging allows multiple readers to access the database while a single writer commits changes. Without WAL, every write locks the entire file, causing “database is locked” errors when the viewer and a script try to read simultaneously.

Single-Writer Rule

Only the Orchestrator (or an agent explicitly delegated by the Orchestrator) issues INSERT, UPDATE, or DELETE against translations.db. All other agents read only. This is a convention, not a technical enforcement, but breaking it leads to contention and potential corruption.

Backup Protocol

Before any write operation, the database is backed up with a timestamp:

backup/auto-backups/YYYY-MM-DD/database/translations.db.bak.HHMMSS

If WAL files exist, they are backed up too. A dedicated script (scripts/backup_db.py) performs this automatically. The script copies both translations.db and translations.db-wal if present. Backup snapshots are retained for 7 days.

4. File System Architecture

Root Structure

Translations/
  translations.db               -- canonical database
  "Project database.sh"           -- launcher
  "Project database.desktop"      -- double-clickable desktop entry
  launch-background.sh            -- background launcher (no terminal window)
  stop-server.sh                  -- stops the background server

  Translation_inbox/              -- owner drops jobs here
    (one subfolder per job)

  Projects/                       -- all project folders
    ID2026-01-01-001/
      job/
        source/                   -- original client files
        in_progress/              -- working drafts, fixes, proofreading
        target/                   -- FINAL DELIVERABLE ONLY
      context/                    -- reference material
      terminology/                -- glossary files
      temp/                       -- working files for agents
      project_manifest.yaml       -- read-only export from DB
      output_config.json          -- project-level format settings

  viewer/
    index.html                    -- HTML viewer (works offline)
    server.py                     -- tiny HTTP server with DB mutation endpoints

  scripts/
    create_project.py             -- project intake from inbox
    generate_docx.py              -- DOCX deliverable generation
    repo_ingest.py                -- corpus + terminology + QMD update
    backup_db.py                  -- timestamped DB backup
    terminology_query.py          -- search terminology entries
    terminology_approve.py        -- approve pending terminology (legacy)
    ingest_termbase.py            -- manual curated termbase ingest
    export_manifest.py            -- export project manifest from DB
    install_dependencies.sh       -- one-shot dependency installer

  corpus/                         -- exported whole-document text files
  templates/                      -- DOCX templates for deliverables

File Classification Rules

Every file entering the system is classified into one of eight types. The File Handler applies these rules in order; the first matching rule wins:

The owner can override classification by placing a manifest_override.yaml in the inbox folder:

overrides:
  - file: "notes.txt"
    classify_as: "context"
  - file: "glossary.csv"
    classify_as: "terminology"

Word Counting per Format

The File Handler computes word counts using format-appropriate tools:

Total word count is stored in projects.word_count; per-file counts go to project_files.word_count.

Naming Conventions

• Project folder: Projects/IDYYYY-MM-DD-XXX where XXX is a sequential daily counter (001, 002, ...). The script reads existing folders to determine the next number.
• Target files: _. — e.g. Metaphysics of speech_pl.odt. Spaces are preserved.
• Versioned working files: When a draft is revised, the filename itself carries the version: draft_v1.txt, draft_v2.txt.

The job/in_progress/ vs. job/target/ Distinction

This is the most important rule in the file system. Only files in job/target/ are deliverables. Everything else is a working version.

output_config.json

Generated during intake if the source format suggests a default output format (e.g. PDF sources default to DOCX output). Contains project-level settings:

{
  "default_output_format": "docx",
  "template_path": "templates/basic_translation_template.docx"
}

5. Agent Architecture (Functional Roles)

The system assigns work to five functional roles. Each role has a defined function, personality traits, domain expertise, operating rules, tool requirements, and error reporting protocol. In a student implementation, these roles could be separate scripts, separate user accounts, or simply documented conventions followed by a single operator.

5.1 Orchestrator

Function: Routes tasks, manages state, interfaces with the owner. The Orchestrator is the only role that may write to translations.db. It monitors the inbox on owner command, initiates project creation, assigns work to other roles, advances status, and generates summary reports.

Personality traits: Methodical, patient, conservative about state transitions. Never auto-advances a job past owner-dependent phases without explicit confirmation (with one exception: the 3-cycle AI review loop guard).

Domain expertise: Higher education administration, Translation Studies workflow design, project management conventions.

Operating rules:

• Only role with write access to translations.db.
• Must create a timestamped backup before any write.
• Must confirm file classification with the owner when ambiguity exists.
• Must record all state transitions with timestamps.
• Must not generate invoices; only stores references to externally generated invoices.
• Must ask for job type (translation / proofreading / other) as the first question on every new job.
• Must collect delivery date before finalising the project record.

Tools needed: SQLite access (read/write), Bash (folder scanning, file moves), YAML parsing.

Error reporting: Halts and reports to owner on database lock failures, missing inbox folders, or unresolvable file classification conflicts.

5.2 Translator

Function: Performs the actual translation. Polish↔English specialist. Adapts register, terminology, and audience. Works from placeholder-masked source text and restores non-translatable elements after translation.

Personality traits: Linguistically conservative, terminology-aware, register-sensitive. Asks questions when requirements are unclear rather than guessing.

Domain expertise: Polish-English linguistics, institutional register adaptation (British English default), domain-specific terminology conventions, CAT tool conventions, terminology management, sworn translation requirements.

Operating rules:

• Must not translate text inside {{PHn}} placeholders.
• Must reproduce placeholders verbatim in the target text.
• Must save drafts to job/in_progress/ only.
• Must register every output file in project_files.
• Must consult terminology/ files before translating.
• Must respect the English variant specified in requirements (British, American, or neutral).

Tools needed: Plain-text editing, format conversion tools (libreoffice, pandoc, pdftotext), terminology query access.

Error reporting: Reports to Orchestrator when source text is ambiguous, when placeholders appear malformed, or when terminology conflicts arise.

5.3 AI Reviewer / Proofreader

Function: Reviews translation quality using the MQM framework. Also handles standalone proofreading jobs (no MQM scoring, just error detection). Operates after the Translator produces a draft and before owner proofreading.

Personality traits: Rigorous, detail-oriented, sceptical of the Translator's output. Separates translation quality from translation effort.

Domain expertise: MQM typology (seven dimensions), severity weighting, Polish-English error patterns, domain-specific translation norms (academic, legal, medical, technical, and others).

Operating rules:

• Must not be the same agent instance that performed the translation (separation of duties).
• Must load source and draft side by side.
• Must score all seven MQM dimensions.
• Must flag critical errors immediately and halt progression.
• Must write a human-readable report to reviews/mqm_review.md.
• Must respect the 3-cycle loop guard.

Tools needed: MQM scoring rubric, side-by-side text comparison, database read access.

Error reporting: Reports critical errors directly to Orchestrator with dimension, severity, and segment reference.

5.4 File Handler / Cataloguer

Function: File operations, word counting, format conversion, database writes (under Orchestrator delegation). Classifies files, computes checksums, counts words per format, moves files between folders.

Personality traits: Pedantic about paths, defensive about data loss, never overwrites without confirmation.

Domain expertise: File systems, text extraction pipelines, SHA-256 integrity checking, Python scripting.

Operating rules:

• Must compute SHA-256 checksums for all source files.
• Must count words using format-appropriate methods (see Section 4.3).
• Must create folder structure before any file moves.
• Must move files only after successful database commit.
• Must handle path quoting correctly (no backslash-escaped spaces).

Tools needed: wc, libreoffice, pdftotext, pandoc, python-docx, openpyxl, odfpy, SHA-256, SQLite (under delegation).

Error reporting: Reports extraction failures, unrecognised formats, and checksum mismatches.

5.5 Ingest / Repository Manager

Function: Corpus and terminology ingest, QMD index updates. Runs after a project reaches DONE status. Extracts source and target text, stores whole-document blobs, populates segment cache, manages curated terminology, updates semantic search index.

Personality traits: Batch-oriented, idempotent, tolerant of partial failure.

Domain expertise: Text alignment heuristics, QMD semantic search, corpus linguistics, terminology curation.

Operating rules:

• Must store whole documents as blobs (no paragraph alignment required).
• Must deduplicate terminology before insert (normalise case, strip spaces).
• Must run QMD update after ingest; if QMD is unavailable, proceed with flag QMD_SKIP.
• Must log all actions in audit_log.

Tools needed: pandoc, pdftotext, QMD binary, SQLite (under delegation).

Error reporting: Reports corpus extraction failures, QMD binary absence, and terminology parse errors.

6. Translation Workflow (Methodology)

The pipeline has phases. Each phase is labelled with the design increment in which it was introduced.

Job Type Fork (Determines Which Phases Run)

Before any work begins, the Orchestrator asks the owner: “What type of job is this?” The answer determines the active pipeline.

The functional roles in the table below are generic. Assign your own names when building the system.

Proofreading jobs skip the TRANSLATING phase because the source text is treated as the owner’s draft. The AI reviewer evaluates the existing text directly.

Phase 1: Ingest & Cataloguing (Increment 1)

1. Owner drops files. One subfolder per job inside Translation_inbox/. Mixed files (source + context + terminology) are allowed.
2. Folder scan. Orchestrator lists the inbox. Hidden files and readme.md are ignored.
3. File classification. File Handler applies the six classification rules. If ambiguity exists, the Orchestrator presents a numbered file list to the owner: “A = job, B = context, C = terminology.”
4. Override check. If manifest_override.yaml exists in the inbox folder, the File Handler applies owner-specified overrides before computing word counts.
5. Word count. File Handler counts words using format-appropriate methods (see Section 4.3). Total stored in projects.word_count; per-file counts in project_files.word_count.
6. Rate confirmation. Orchestrator asks owner for rate per word, calculates price: price = word_count × rate_per_word. Owner confirms or overrides.
7. Client resolution. Orchestrator asks: “Who is the client?” New clients are inserted into clients; existing ones are reused.
8. Language pair confirmation. Orchestrator detects source language or asks owner. Records language_pair (EN-PL or PL-EN) and direction.
9. Job type confirmation. Orchestrator asks: “Translation (A), proofreading (B), or other (C)?” Records projects.job_type.
10. Delivery date. Owner provides deadline. Stored in projects.delivery_date. The system does not auto-alert, but the owner can query via the viewer.
11. Database write. File Handler creates timestamped backup, then inserts projects, project_files, and clients rows. Status = ACTIVE, sub_status = IDENTIFYING.
12. Move & cleanup. Files copied to project folder. Inbox subfolder deleted.
13. Manifest export. Orchestrator writes project_manifest.yaml as a read-only export.

Phase 2: Requirements Gathering (Increment 1)

This phase is a hard gate. The Orchestrator cannot advance to translation work until the owner has answered every question.

1. Orchestrator briefs Translator with source file path, format, client, and context documents.
2. Translator scans source + context and drafts a requirements questionnaire:

• Job type (translation / proofreading / other) — already recorded; confirmed here
• Domain (academic / administrative / certified / general / legal / literary / marketing / medical / technical)
• Audience (general public / academic / legal / medical)
• Register (formal / informal / technical)
• English variant (British / American / neutral)
• Special conditions (certified translation, sworn translator note, word-count limit, publication-ready)
• Delivery format preference (DOCX, ODT, PDF)
• Delivery date confirmation

3. Orchestrator presents questionnaire to owner. Owner replies in chat.
4. Orchestrator updates projects row: delivery_date, summary, domain, sub_status = TRANSLATING (for translation jobs) or MQM_REVIEW (for proofreading jobs).

Phase 3: Translation (Increment 1)

Runs only for job_type = 'translation'.

1. Context retrieval. Before drafting, the Translator queries QMD using the first 500 words of the source text. Results are cached in temp/[project_key]_tm_suggestions.md and inform register, terminology, and style choices. Per-segment QMD queries are rejected as unworkable — they add latency without proportional benefit.
2. Segmentation. Translator converts source to plain text, segments by paragraph (blank-line delimited), and translates each segment.
3. Draft write. Translator writes translated text. For .odt, writes plain text and Orchestrator runs libreoffice --headless --convert-to odt.
4. Save. File placed in job/in_progress/ with correct naming, registered as file_type = 'draft'.
5. After draft: sub_status -> MQM_REVIEW (status becomes REVIEW).

Phase 4: AI LQA Review (Increment 2)

Runs for both translation and proofreading jobs.

1. AI Reviewer loads source + draft side by side.
2. Runs MQM assessment across all seven dimensions using fixed-deduction scoring.
3. Writes review results to mqm_reviews table and a human-readable report to Projects/{key}/reviews/mqm_review.md.
4. If score >= threshold and no critical errors: status -> ACTIVE, sub_status -> PROOFREADING.
5. If score < threshold or critical errors exist: report sent to owner; sub_status -> FIXING (status remains ACTIVE).

Phase 5: Fix Implementation (Increment 2)

Runs only when the AI reviewer flags issues.

1. Orchestrator presents AI review report + owner authorisation to Translator.
2. Translator edits draft in place or creates revised copy in job/in_progress/. New file registered as file_type = 'fixed'.
3. After fixes, sub_status -> MQM_REVIEW (for re-check) or PROOFREADING (if owner waives re-check).
4. Loop guard: If this is the third AI review cycle, sub_status is forced to PROOFREADING with flag MQM_CYCLES_EXHAUSTED.

Phase 6: Final Proofreading & Sign-off (Increment 1)

This phase is a hard gate.

1. Orchestrator notifies owner: “Draft is ready for proofreading.” Project is in ACTIVE / PROOFREADING. No timeout.
2. Owner reviews draft. Changes may be made directly (saved as proofreading file) or relayed to Translator.
3. Owner sign-off. Orchestrator asks: “Have you finished proofreading? Do you accept the text as final?”

• If owner confirms: Orchestrator records owner_sign_off = 'OWNER_APPROVED' timestamp, moves approved file from job/in_progress/ to job/target/, retags as file_type = 'target'. Status -> DONE, sub_status -> CLOSED.
• If owner requests changes: return to AI review (AI Reviewer revises). Maximum 3 AI review cycles, then force advance.
• owner_sign_off is the legal traceability record. A project cannot reach CLOSED without it.

Phase 7: Repository Ingest (Increment 3)

1. Corpus ingest. Ingest Manager reads source and target files, stores as whole-document blobs in corpus_entries.
2. Segment cache. Paragraphs are naively aligned; sentence-level alignment is attempted only when paragraph lengths match. Segments stored in segment_cache.
3. Terminology curation. No automatic extraction runs at ingest time. The terminology bank is maintained through manual ingest_termbase.py runs (see Section 8.3).
4. QMD update. Orchestrator runs qmd update to refresh the semantic index. If QMD is unavailable, proceeds with flag QMD_SKIP.

Phase 8: Project Summary & Close (Increment 1)

1. Orchestrator generates summary report: project key, client, language pair, delivery date, word counts, MQM score (if applicable), corpus/terminology additions (if applicable).
2. Report delivered to owner.
3. Manifest archived.

State Transition Diagram

                  +-----------+
                   |  PENDING  |
                   +-----+-----+
                         | owner drops files
                         v
                 +-----------------------+
                 |        ACTIVE         |
                 |  sub: IDENTIFYING     |<-------------------------+
                 +-----------+-----------+                        |
                             | classification complete            |
                             v                                    |
                 +-----------------------+                        |
                 |        ACTIVE         |                        |
                 |  sub: REQUIREMENTS    |                        |
                 +-----------+-----------+                        |
                             | requirements confirmed             |
                             v                                    |
                 +-----------------------+                        |
                 |        ACTIVE         |                        |
                 |  sub: TRANSLATING     |                        |
                 +-----------+-----------+                        |
                             | draft complete                     |
                             v                                    |
        +--------------------+--------------------+               |
        |                                         |               |
        | job_type = translation                  | job_type = proofreading
        v                                         v               |
   +-----------------------+      +-----------------------+       |
   |        REVIEW         |      |        REVIEW         |       |
   |  sub: MQM_REVIEW      |      |  sub: MQM_REVIEW      |       |
   +-----------+-----------+      +-----------+-----------+       |
               |                                |                |
               | score OK                       | score OK       |
               v                                v                |
       +-----------+                  +-----------------------+   |
       |    DONE   |                  |        ACTIVE         |   |
       |  CLOSED   |                  |  sub: PROOFREADING    |---+
       +-----------+                  +-----------+-----------+
                                               ^
                                               | fixes needed
                                      +-----------+-----------+
                                      |        ACTIVE         |
                                      |  sub: FIXING          |
                                      +-----------------------+

Note on MQM_REVIEW: the workflow discussions refer to this phase as “AI_REVIEW”. The database CHECK constraint still contains the original MQM_REVIEW string. The two names are synonymous in practice.

7. Quality Assurance: MQM Framework

What Is MQM?

Multidimensional Quality Metrics (MQM) is an industry-standard error typology for translation quality assessment. Unlike holistic “good/bad” scoring, MQM breaks quality down into dimensions, each checked for errors classified by severity. The system uses MQM Core, which defines seven high-level dimensions.

The Seven Dimensions

Fixed-Deduction Scoring Model

The system uses a fixed-deduction model: each error deducts a flat penalty regardless of document length. This makes scores comparable across jobs.

Per-dimension score:

dimension_penalty = (critical_count * 10) + (major_count * 5) + (minor_count * 1)
dimension_score = max(0, 100 - dimension_penalty)
overall_score = mean of seven dimension scores

Example: a 100-word text and a 1,000-word text each with 1 major error both score 95 in that dimension.

Threshold Policy

A default threshold of 90/100 is documented as a placeholder. It is not enforced automatically until at least five sample translations have been scored by a human calibrated rater and the owner confirms the threshold is appropriate for Polish-English institutional translation (calibrate per domain). Until then, the AI reviewer reports the score and flags critical errors, but the Orchestrator advances the job based on owner command, not the score alone.

Chunked Review Strategy

For documents exceeding 5,000 words, the AI reviewer reviews in chunks of ~2,000 words to maintain focus. Each chunk receives a provisional score; the overall score is the mean of chunk scores.

Loop Guard

The AI fix loop (MQM_REVIEW <-> FIXING) is capped at 3 cycles. After the third cycle, the job is forced to ACTIVE / PROOFREADING with flag MQM_CYCLES_EXHAUSTED. This prevents oscillation when the Translator and AI reviewer disagree on error severity.

8. Corpus & Terminology Repository

The TMS provides three levels of corpus reuse, in order of priority:

1. QMD semantic search (primary) — conceptual similarity across whole documents.
2. Whole-document corpus browsing (secondary) — manual verification of previous translations in context.
3. Segment cache exact-match (tertiary fallback) — verbatim phrase reuse only.

Repository Tables

8.1 Corpus Architecture

The system stores completed translations as whole-document blobs in corpus_entries. This is a deliberate choice:

• Whole-document blobs preserve paragraph and formatting context. A translator searching the corpus sees the surrounding text, not an isolated sentence.
• No paragraph alignment is attempted in the baseline design. Paragraph-level alignment is unreliable across Polish and English because the two languages have different paragraph conventions.
• Sentence alignment is attempted heuristically during repository ingest only when source and target paragraphs have identical sentence counts. Otherwise, the paragraph is stored as a single segment.
• Domain filtering. The domain_tag column enables domain-filtered corpus queries. A translator working on a medical text can restrict corpus search to previous medical jobs. See Section 1 for the design rationale behind the context-first architecture.

8.2 QMD Integration for Semantic Search

QMD (a semantic search tool) indexes the corpus for conceptual similarity. After every repository ingest, the Orchestrator runs:

PATH="/path/to/qmd:$PATH" qmd update

During translation (Increment 3), the Translator queries QMD using the first 500 words of the source text. Results are cached in temp/[project_key]_tm_suggestions.md. Per-segment queries during translation are rejected as unworkable — they add latency without proportional benefit.

8.3 Terminology Curation Workflow

The terminology bank is manually curated. There is no automatic extraction pipeline.

How entries are added:

1. The owner designates one or more termbase source files (e.g. general_glossary.txt, medical_terms.xlsx).
2. The owner runs ingest_termbase.py with --source-file, --domain, and --source-name arguments.
3. The script parses the file, deduplicates against existing entries (case-insensitive match on source_term + target_term + language_pair), and inserts new rows with approved = TRUE.
4. Entries longer than 100 characters are flagged for owner review and written to a review_*_long_terms.txt file.
5. Malformed entries (for .txt parses with unexpected = counts) are written to review_*_malformed.txt.
6. Immediately after ingest, the entries are available to Translator and AI reviewer queries.

Key design decisions:

• Single-direction storage. Only EN→PL pairs are stored. The viewer and query scripts search both source_term AND target_term, so a Polish query still finds the match. This avoids doubling the table size.
• Bidirectional search. The viewer's termbase filter and the search-terms endpoint both match the query against source_term OR target_term using LIKE.
• Domain filtering. Every query can include a domain parameter. The CHECK constraint permits: academic, administrative, certified, general, legal, literary, marketing, medical, technical.
• Provenance tracking. The source_file column records which ingest run produced the entry (e.g. “general_glossary.xlsx”, “medical_terms.txt”).
• No approval button. Because all entries enter via manual curated ingest, there is no “Approve” button in the viewer. The actions are: Save (edit in place), Delete (permanent removal).
• Per-page pagination. The viewer supports 50, 250, or 500 results per page.

Script usage example:

python3 \
    scripts/ingest_termbase.py \
    --source-file "terminology/general_glossary.txt" \
    --domain general \
    --source-name "general_glossary"

Supported formats: .xlsx (two-column style with en/pl headers) and .txt (single-line style with = delimiters and ; variant separators).

8.4 Segment Cache as Lightweight Fallback

The segment_cache table acts as a lightweight translation memory. Because paragraph alignment is unreliable when source and target paragraph counts differ (see Weakness #18), the segment cache is intentionally treated as a best-effort fallback rather than an authoritative translation memory. It stores individual source-target segment pairs extracted during repository ingest. Querying it is simple:

SELECT target_segment FROM segment_cache
WHERE source_segment = ? AND language_pair = ?;

Limitation: The segment cache supports exact-match lookup only. There is no fuzzy matching. If the source sentence differs by even one word, the cache returns nothing. This is a known limitation (see Section 12).

9. Comparison: TMS vs. Classic TM/CAT Systems

The table below summarises the feature-level differences. What follows is a domain-by-domain argument for when the TMS offers capabilities that commercial CAT tools cannot replicate, at a cost commercial tools cannot match.

Customisation is not a feature — it is the architecture. Every component is exposed for modification because the system assumes no two freelancers have identical needs.

The TMS intentionally substitutes whole-document semantic search for fuzzy TM. See Section 1, “A different paradigm.”

Domain-by-Domain Argument

Academic

Academic translation lives or dies on register consistency and citation integrity. A CAT tool will match “hermeneutic circle” from a previous job, but it will not warn you that the current client's style guide prefers “hermeneutic cycle.” The TMS requirements phase captures the style guide explicitly; the AI reviewer checks against it during the MQM Style dimension. Placeholder protection ensures that LaTeX citations, DOI strings, and bibliography keys survive translation untouched. The corpus stores whole documents, so a translator can verify whether they used the same rendering of “performative utterance” in a paper six months ago. For Polish↔English academic work, the AI review loop catches register drift and citation corruption that CAT QA modules do not even attempt to detect.

The honest limitation: the TMS has no fuzzy matching. If the source text contains “performative utterances” (plural), the exact-match segment cache returns nothing. The translator must recognise the variant manually or rely on corpus search.

Administrative

University faculties, HR departments, and student offices produce high-volume, repetitive document streams: council minutes, Erasmus agreements, degree confirmations. Speed matters, but so does consistency — “Rector” must not become “Vice-Chancellor” halfway through a reporting cycle. The TMS handles this through template-based DOCX generation, batch intake, and the segment cache for recurring phrases. The audit log provides the traceability that institutional audits demand. Most importantly, the zero licensing cost means a language centre can deploy the system without begging the bursar for a software budget.

The honest limitation: the TMS does not support real-time multi-user collaboration. If two staff members try to update the same project simultaneously, SQLite's file locking serialises the writes. A convention — one designated Orchestrator operator — prevents conflicts.

Certified / Sworn

Certified translations carry legal liability. A mistranslated date of birth on a sworn diploma or a misidentified party in a notarised contract can invalidate the document and expose the translator to disciplinary proceedings. The TMS treats MQM Critical errors as a hard gate: the AI reviewer halts the pipeline until the error is fixed. The owner_sign_off timestamp is a legal traceability record: the owner confirmed the final text before delivery. The audit log records every change with agent name and timestamp. No commercial CAT tool provides a built-in legal traceability layer of this granularity.

The honest limitation: placeholder detection is regex-based. Unusual formatting or non-standard citation styles are missed. Lowercase proper names (e.g. “iPhone”) may be translated in error. The owner must review temp/[key]_placeholders.json before translation.

Legal

Legal translation demands precision for liability reasons, but it also demands consistency across document families. A lease agreement and its addendum must use identical terminology for “force majeure” and “termination clause.” The TMS terminology bank stores these terms with domain tags, and the bidirectional search ensures that a Polish query finds the English equivalent even though only the EN→PL direction is stored. The AI reviewer loads a legal severity rubric during the requirements phase, weighting Critical errors more heavily for financial and contractual terms. Corpus search lets a translator verify how they rendered “indemnification” in a previous mandate.

The honest limitation: the TMS does not generate sworn-translator clauses or certification stamps automatically. The requirements phase must capture this need, and the owner must insert the clause manually.

Literary

Literary translation is the domain where CAT tools are weakest and the TMS is strongest. A CAT tool treats “He drained his glass” as a reusable segment; the TMS treats it as a phrase whose register and emotional colouring must match the protagonist's voice across 400 pages. Corpus search — especially QMD semantic search — lets a translator check whether the voice was consistent across chapters translated months apart. The chunked AI review strategy handles long novels in 2,000-word slices without reviewer fatigue. The requirements phase captures audience age, genre conventions, and cultural adaptation needs.

The honest limitation: the TMS has no sentence alignment. Paragraph-level alignment is coarse. A translator searching the corpus for a specific sentence may receive the entire paragraph. For literary work, this is often a feature (context is everything), but it wastes storage.

Marketing

Marketing translation is about persuasion, not fidelity. A slogan that works in Polish may be nonsense in English if translated literally. The TMS requirements phase captures brand guidelines, tone-of-voice documents, and competitor positioning. The AI reviewer checks the Style and Audience Appropriateness dimensions against these guidelines. The segment cache is less useful here — marketing copy rarely repeats verbatim — but the corpus search lets a translator check how the brand spoke about itself in previous campaigns.

The honest limitation: the TMS does not integrate with design tools (InDesign, Figma). Format preservation handles text files only; desktop publishing workflows are out of scope.

Medical

Patient safety makes medical translation the most liability-sensitive domain in this list. A single mistranslated drug dosage, anatomical reference, or contraindication can cause serious harm. The AI reviewer treats every Critical error as a pipeline halt. Your curated terminology bank stores domain-tagged entries from your chosen termbases — for example, a general-domain glossary and a specialised medical glossary. A query for “appendix” returns different results in the medical domain (vermiform appendix) and the general domain (document appendix) because the domain filter routes the query correctly. The MQM Terminology dimension is weighted most heavily for medical jobs.

The honest limitation: the terminology bank does not include drug interaction data or dosage conversion tables. It is a linguistic resource, not a clinical decision support system. The translator — not the TMS — bears final responsibility for medical accuracy.

Technical

Technical translation lives on terminology precision and string reuse. Software strings, user manuals, and safety instructions repeat across product versions. The segment cache serves as a lightweight TM for exact-match reuse. Placeholder protection ensures that variables, code snippets, and UI element IDs survive translation untouched. The requirements phase captures the product's terminology conventions and UI style guide.

The honest limitation: the segment cache is exact-match only. If a software string changes from “Save file” to “Save document,” the cache returns nothing. There is no fuzzy matching for near-misses. For high-volume software localisation, a dedicated CAT tool with fuzzy TM is more efficient.

General

General-domain translation — letters, emails, casual web content — is where the TMS is least differentiated from a CAT tool. The requirements phase is lighter, the MQM threshold is lower, and the segment cache provides modest benefit because general texts rarely repeat. Where the TMS still wins is cost: a freelance translator doing occasional general work cannot justify a Trados licence. The TMS provides terminology search, corpus lookup, and audit logging at zero licensing cost.

The honest limitation: for pure general-domain volume work, the TMS's agent-compute overhead may not be worth the quality gain. If the client cares only about speed and cost, raw MT with minimal post-editing is cheaper.

10. Strengths & Weaknesses

Strengths

1. Zero licensing cost. Every tool is open source or already installed on a standard Linux workstation.
2. Full audit trail. Every action is logged in audit_log with timestamp, agent, and details.
3. Quality enforcement. AI review loop prevents low-quality translations from reaching clients.
4. Placeholder protection. Non-translatable elements (proper names, codes, citations, numbers) are automatically masked and restored.
5. Flexible file support. Eleven formats handled through a unified pipeline.
6. Semantic corpus search. QMD integration allows conceptual similarity search across completed translations.
7. Curated terminology. Manually approved entries from your chosen termbases, domain-filtered, bidirectionally searchable.
8. Job type flexibility. Translation, proofreading, and custom jobs in one system.
9. Incremental build. Gall's Law satisfied: the prototype worked before complexity was added.
10. Full source code ownership. Every script is readable, modifiable Python. No vendor lock-in.

Weaknesses

11. No fuzzy matching. The segment cache is exact-match only. A translator searching for “university budget” gets nothing if the cache contains “university budgets.” For Polish-English institutional documents, QMD semantic search was found to outperform fuzzy TM, making this a deliberate trade-off rather than an unplanned gap.
12. No sentence alignment. Paragraph-level alignment is coarse. A true TM needs sentence-level alignment, which this system does not reliably provide.
13. Single-user. SQLite and file-system locking assume one writer. Multi-user scenarios require architectural change.
14. No invoice generation. The system stores invoice references (invoice_pdf_path, invoice_number) but does not generate invoices. Accounting is external.
15. No payment tracking beyond paid/unpaid. No partial payments, no overdue alerts, no client balance.
16. QMD dependency is brittle. If the QMD binary is missing or the Bun runtime is uninstalled, semantic search fails silently.
17. Placeholder detection is regex-based. Unusual formatting or non-standard citation styles are missed. Lowercase proper names (e.g. “iPhone”) may be translated in error.
18. Chunk misalignment in long documents. The naive paragraph alignment in repo_ingest.py can pair the wrong paragraphs when source and target paragraph counts differ.

11. Use Cases

Academic papers

Journal submissions, research grant proposals, conference abstracts. The system handles LaTeX-like citation syntax protection and academic register adaptation. The same pipeline handles legal contracts, medical consent forms, and technical manuals — the requirements phase loads the appropriate register and terminology set.

Administrative forms

Faculty council minutes, HR documents, student records. These often require certified or sworn translation notes; the requirements-gathering phase captures this.

Conference abstracts

Short, high-visibility texts where terminology accuracy is critical. AI review is strongly recommended.

Book chapters

Long-form texts where chunked AI review and corpus reuse provide the most benefit.

Certified documents

Diplomas, transcripts, legalised documents. The system records sworn translator notes and certification requirements in projects.summary.

Post-editing

Machine-translated texts brought in for human refinement. The source is treated as a draft file type; the Translator performs post-editing rather than translation from scratch.

Proofreading-only jobs

Owner provides a completed translation for AI review and final polish. The job_type = 'proofreading' path skips the translation phase.

12. Limitations & Known Issues

13. Browser & Viewer Architecture

Why HTML/JS, not a native GUI?

The viewer is a single HTML file (viewer/index.html) that uses sql.js (a JavaScript build of SQLite) to load the database entirely in the browser. This choice has three advantages:

1. No build step. The file opens in any modern browser. No Electron, no Qt, no Tkinter.
2. Offline operation. Once loaded, the viewer needs no server. Drag-and-drop works from file:// protocol.
3. Reuse of existing pattern. The team already built Database/Viewer/kb_browser.html for the literature knowledge base. The TMS viewer is an adaptation of that proven design.

Viewer Features

• Auto-load: When served via server.py, the viewer fetches ../translations.db automatically via HTTP.
• Drag-and-drop: When opened from file://, the user drops the .db file into a zone.
• Project overview: Summary card showing total, pending, active, review, and done project counts.
• Filters: Search by project key, client, status, language pair, sub-status. All dropdowns are sorted alphabetically.
• Status badges: Color-coded pills (PENDING = yellow, ACTIVE = green, REVIEW = blue, DONE = grey).
• File-type pills: Each file in a project detail panel is tagged with a color-coded pill (source, draft, mqm_reviewed, fixed, proofreading, target, context, terminology).
• Collapsible Edit Project section: Click “Edit Project ▼” to expand editable fields. All changes are saved via a single unified Save button.
• Editable fields: Client (dropdown, sorted), Domain (dropdown, sorted), Paid Status (dropdown), Invoice Number, Delivery Date, Translator, Reviewer.
• Corpus view: Browse whole-document corpus entries with source/target word counts, domain tags, and expandable previews. Domain tags are editable inline when server.py is running.
• Termbase view: Search against source_term OR target_term (bidirectional), filter by domain, filter by source_file (provenance), filter by approval status, paginate at 50/250/500 results per page.
• Icon-based actions: ✓ Save (persist edits), ✕ Delete (permanently remove row). No Approve button — curated entries enter already approved.
• Polish number formatting: The viewer uses Polish-style number formatting (spaces as thousands separators). Change plNum() in index.html if your locale uses commas.
• /open-folder endpoint: Clicking a file path opens the containing folder in the system file manager via xdg-open.

Server Architecture

viewer/server.py is a tiny Python http.server subclass with additions:

1. CORS headers: Allow the viewer to load the database cross-origin during development.
2. /open-folder endpoint: Accepts a path query parameter, validates it is inside Translations/, and opens it with xdg-open.
3. /update-project endpoint: Persists edits to projects rows (client, domain, paid status, invoice number, delivery date, translator, reviewer).
4. /edit-term endpoint: Updates terminology fields (source_term, target_term, part_of_speech, domain, context_note).
5. /delete-term endpoint: Permanently removes a terminology row.
6. /update-corpus endpoint: Updates corpus domain tags.
7. /search-terms endpoint: Returns paginated, filtered terminology results as JSON.
8. /pair-terms endpoint: Bulk-inserts terminology pairs from a project's candidate list.

The launcher script (Project database.sh) starts the server, records its PID, opens the browser, and tails the log file.

14. Required Tools & Plugins

System Tools

Python Libraries

Search / Index Tools

Browser Requirements

Any modern browser (Chrome, Firefox, Edge, Safari). The viewer uses vanilla JavaScript with no frameworks. sql.js ASM build is included as a vendor file; no WASM file is needed, so file:// protocol works.

15. Implementation Roadmap

Increment 1: Skeleton (Week 1)

Goal: A working system that can ingest a job, track it in a database, and display it in a viewer.

• Create translations.db with clients, projects, and project_files tables.
• Build folder structure (job/source, job/in_progress, job/target, context, terminology).
• Implement file classification rules and owner override (manifest_override.yaml).
• Adapt kb_browser.html → viewer/index.html (list projects, filter by status).
• Build project intake script (create_project.py): detect inbox folder, create project folder, write DB row, move files.
• Validation: Drop a .txt file in the inbox; run intake; viewer shows the project.

Increment 2: Translation Loop + AI Review (Week 2)

Goal: End-to-end translation with quality gating.

• Add mqm_reviews table.
• Implement requirements-gathering questionnaire with job type fork.
• Translator produces draft from source text.
• Onboard AI Reviewer role with MQM scoring rubric.
• Implement fixed-deduction scoring with placeholder threshold.
• Implement fix loop with 3-cycle guard.
• Add state machine enforcement in Orchestrator.
• Validation: Submit a Polish text; receive an English draft and an MQM report.

Increment 3: Corpus + QMD + Termbase (Week 3)

Goal: Reusable repository, semantic search, and curated terminology.

• Add corpus_entries, terminology_entries, audit_log, and segment_cache tables.
• Store whole documents as blobs; attempt naive paragraph alignment.
• Build ingest_termbase.py for manual curated terminology ingest.
• Integrate QMD batch query (one per project, first 500 words of source).
• Add optional cron trigger for inbox polling.
• Refine MQM threshold based on calibrated human scores.
• Validation: Complete a project, query QMD for a previously translated phrase, receive a match.

Post-launch

• Critique the workflow document for structural weaknesses.
• Audit system files (schema, viewer, scripts) for correctness.
• Document edge cases discovered during live operation.

16. Pedagogical Notes

Suggested Exercises

1. Schema modification. Ask students to add an escalation_at column to projects and implement a reminder script that notifies the owner when a job has been in REQUIREMENTS for more than 24 hours.
2. Format handler. Add support for .epub files by writing a new word-count function in create_project.py and registering the format in the CHECK constraint.
3. MQM calibration. Provide five sample translations with known errors. Have students score them using the MQM rubric, compare results, and agree on a project-specific threshold.
4. Corpus query. Write a Python script that queries segment_cache for exact matches and reports recall (what percentage of a new source text has been translated before).
5. Viewer extension. Modify viewer/index.html to display a pie chart of projects by status using Canvas or SVG.
6. Termbase ingest. Provide a small .txt termbase file with malformed entries. Have students run ingest_termbase.py, inspect the review files, and fix the source data.

Discussion Questions

7. Why does the system use two state columns (status and sub_status) rather than one? What problems would a single 10-state column create?
8. The AI Reviewer cannot be the same agent instance that translated the text. Why? What psychological or methodological principle does this enforce?
9. Whole-document blobs in corpus_entries waste storage compared to aligned segments. Why did the designer choose blobs anyway?
10. The system has no fuzzy matching. For a language pair like Polish-English, where inflection is rich, how much value does exact-match TM actually provide?
11. The terminology bank stores only EN→PL pairs, yet PL→EN search works. What are the trade-offs of this single-direction storage design?
12. Owner sign-off is a hard gate before CLOSE. In a real workflow, what legal or professional risk does this mitigate?
13. The job_type column introduces workflow forks. What other job types might a small language centre need, and what schema changes would they require?