BA-translator/README.md

BA-translator

A powerful tool for extracting and applying translations from FlatBuffers-based SQLite databases

Overview

BA-translator is a comprehensive localization tool designed for games and applications that use FlatBuffers-serialized data stored in SQLite databases. It provides a complete workflow for extracting translatable strings, managing translations through CSV files, and applying them back to the database.

Key Features

• Extract translatable strings from FlatBuffers-based SQLite databases to JSON format
• Export translations to CSV format for easy collaboration with translators
• Import completed translations from CSV back to JSON
• Apply translations to the database with automatic backup creation
• Validate translation consistency and detect potential issues
• Merge multiple CSV files from different translators
• Generate FlatBuffers schemas from C# source files
• Create repository mappings for database table relationships

System Requirements

Required Dependencies

• Python 3.7+ (Python 3.8+ recommended)
• sqlite3 (included with Python)
• flatbuffers - for binary data serialization
• tqdm - for progress bars
• unidecode - for text processing
• flatc - FlatBuffers compiler (for generating Python modules from .fbs schemas)

Required Files

• ExcelDB.db - The main SQLite database file
• repository_map.json - Generated mapping file (created by dumpdbschema.py)
• types.cs - C# source file with type definitions (for schema generation)
• flatc.exe - FlatBuffers compiler executable (Windows) or flatc (Linux/macOS)

Installation

1. Clone or download this repository:

   git clone <repository-url>
   cd BA-translator
   

2. Install Python dependencies:

   pip install -r requirements.txt
   

   Or install manually:

   pip install flatbuffers tqdm unidecode
   

3. Prepare required files:

   • Place ExcelDB.db in the project root directory
   • Place types.cs (decompiled C# file) in the project root

4. Generate repository mapping:

   python dumpdbschema.py
   

   This creates repository_map.json which maps database tables to their schemas.

5. Set up FlatBuffers compiler (if not installed):

   • Windows: Run setup_flatc.bat to automatically download flatc.exe
   • Linux/macOS: Install FlatBuffers: sudo apt install flatbuffers-compiler or brew install flatbuffers

6. Generate FlatBuffers schemas (if needed):

   python parser.py
   

   This creates generated_schema.fbs from C# types.

7. Generate Python modules from schemas:

   python generate_flatbuffer_folders.py --auto
   

   This automatically finds .fbs files and generates compatible Python modules.

Quick Start

Basic Translation Workflow

1. Extract translatable strings:

   python BAtranslator.py extract --output translations.json
   

2. Export to CSV for translators:

   python BAtranslator.py export_csv --input translations.json
   

   This creates two files:

   • translations.csv - Complete file for merging
   • translations_for_translators.csv - Simplified file for translation work

3. Translate the CSV file using any CSV editor (LibreOffice Calc, Google Sheets, or any text editor)

4. Import completed translations:

   python BAtranslator.py import_csv --input translations_for_translators.csv --output translations_updated.json
   

5. Validate translations (optional but recommended):

   python BAtranslator.py validate_csv --input translations_for_translators.csv
   

6. Apply translations to database:

   python BAtranslator.py patch --input translations_updated.json
   

   This automatically creates a backup (ExcelDB.db.bak) before applying changes.

Advanced Usage

Update existing translations with new content:

python BAtranslator.py extract --output new_translations.json --update-from old_translations.json

Merge multiple translator CSV files:

python BAtranslator.py merge_csv translator1.csv translator2.csv translator3.csv --output merged_translations.csv

Command Reference

Main Translation Commands

Command	Description	Usage
extract	Extract strings from database to JSON	python BAtranslator.py extract --output file.json
patch	Apply translations from JSON to database	python BAtranslator.py patch --input file.json
export_csv	Convert JSON to CSV format	python BAtranslator.py export_csv --input file.json
import_csv	Convert CSV to JSON format	python BAtranslator.py import_csv --input file.csv --output file.json
validate_csv	Validate CSV file consistency	python BAtranslator.py validate_csv --input file.csv
merge_csv	Merge multiple CSV files	python BAtranslator.py merge_csv file1.csv file2.csv --output merged.csv

FlatBuffers Generation Commands

Command	Description	Usage
parser.py	Generate .fbs schema from C# types	python parser.py
dumpdbschema.py	Generate repository mapping	python dumpdbschema.py
generate_flatbuffer_folders.py	Generate Python modules from .fbs	python generate_flatbuffer_folders.py --auto

File Formats

JSON Translation Format

{
  "TableName": {
    "row_id": {
      "field_name": {
        "original": "Original text",
        "translation": "Translated text"
      }
    }
  }
}

CSV Format

Column	Description
GroupID	Identifier for grouping identical texts
Original	Original text to translate
Translation	Translated text (fill this column)
SQLTable	Database table name
RowID	Row identifier
Field	Field name
HasCodes	Indicates special characters present
Codes	Encoded special characters

FlatBuffers Schema Generation

Quick Setup

Generate Python modules from .fbs schema files:

# Setup flatc compiler (Windows)
setup_flatc.bat

# Generate Python modules automatically
python generate_flatbuffer_folders.py --auto

# Or from specific files
python generate_flatbuffer_folders.py schema.fbs

Manual FlatBuffers Setup

1. Install FlatBuffers compiler:

   • Windows: Run setup_flatc.bat or download from FlatBuffers releases
   • Linux: sudo apt install flatbuffers-compiler
   • macOS: brew install flatbuffers

2. Generate Python modules:

   # From C# source (if available)
   python parser.py
   
   # Generate Python modules
   python generate_flatbuffer_folders.py --auto --verbose
   

3. Verify generation:

   # Check integration with BAtranslator
   python -c "from BAtranslator import build_schema_map; build_schema_map(); print('OK')"
   

Troubleshooting

Common Issues

1. "Database file 'ExcelDB.db' not found"

   • Ensure the database file is in the project root directory
   • Check the filename is exactly ExcelDB.db (case-sensitive)

2. "Module not found" errors

   • Run python generate_flatbuffer_folders.py --auto to generate required schema modules
   • Ensure FlatData/ directory contains the generated Python files
   • Check that __init__.py files exist in generated directories

3. "File 'types.cs' not found"

   • Place the decompiled C# file in the project root
   • Run python dumpdbschema.py to generate the repository mapping

4. "flatc not found" or "flatc command failed"

   • Windows: Run setup_flatc.bat to download flatc.exe automatically
   • Linux/macOS: Install FlatBuffers compiler using your package manager
   • Verify installation: flatc --version

5. "Generated modules import errors"

   • Regenerate modules: python generate_flatbuffer_folders.py --auto --clean
   • Check that __init__.py files exist in FlatData directory

License

This project is open source. Please refer to the license file for more details.

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

---

📖 For Russian documentation, see README_RU.md

⚠️ This project is tested in older versions of Blue Archive. It may not work correctly in newer versions. Report me if something wrong.