Skip to content

LNA Book Proc API

This project is a FastAPI service that lets you:

  • Authenticate users with JWT
  • Check service health
  • Upload and process EPUB files into:
  • an HTML “viewer” page (viewer.html) with a navigation sidebar
  • a RAG-ready folder containing extracted text and images
  • Convert extracted text chunks (.txt) into audio (.mp3) with Kokoro and concatenate them

The code is intentionally split into routers (HTTP/API layer) and utilities (file processing / templating / audio generation).

1. High-level architecture

Entry point (FastAPI)

main.py creates the FastAPI app, configures CORS, and mounts routers under their prefixes:

  • app.routers.auth → /auth
  • app.routers.health → /health
  • app.routers.epub → /epub
  • app.routers.audio → /audio

See:

main.py (LNA app repo)
# Include routers
app.include_router(auth.router)
app.include_router(health.router)
app.include_router(epub.router)
app.include_router(audio.router)
...
if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=9000)

CORS

main.py configures permissive CORS:

  • allow_origins=["*"]
  • allow_credentials=True
  • allow_methods=["*"], allow_headers=["*"]

For production, restrict allow_origins to your real frontend URLs.

Authentication / middleware

All “processing” endpoints depend on get_current_active_user (JWT bearer token).

JWT logic is implemented in app/auth.py using a small in-memory “fake” user database (useful for development).

app/auth.py (LNA app repo)
SECRET_KEY = "your-secret-key-here-change-in-production"
...
fake_users_db = {
    "testuser": {
        "username": "testuser",
        ...
        "hashed_password": bcrypt.hashpw("testpassword".encode('utf-8'), bcrypt.gensalt()),
        "disabled": False,
    }
}
...
async def get_current_active_user(current_user = Depends(get_current_user)):
    """Get current active user"""
    if current_user.get("disabled"):
        raise HTTPException(status_code=400, detail="Inactive user")
    return current_user

EPUB processing pipeline

The EPUB endpoint (/epub/proc) calls process_epub(...) in app/utils/utils.py.

In practice this pipeline is:

  1. Save upload to a temp directory
  2. Unzip EPUB
  3. Locate the OPS/OEBPS/EPUB folder
  4. Parse content.opf / contents.opf to extract:
  5. the book title + navigation items
  6. structured RAG text chunks (.txt) and a copy of images
  7. Copy certain extracted folders to make assets available to the HTML viewer
  8. Generate/overwrite viewer.html by injecting nav + audio filename into html/viewer_template1.html

See the endpoint and the call into the processing function:

app/routers/epub.py (LNA app repo)
@router.post("/proc", response_model=EpubProcResponse)
async def epub_proc(
    file: UploadFile = File(...),
    book_name: str = Form(...),
    book_id: str = Form(...),
    current_user = Depends(get_current_active_user)
):
    ...
    # Save uploaded file to temp directory
    temp_dir = tempfile.mkdtemp()
    ...
    buffer.write(content)
    ...
    # Process the EPUB file
    result_path, rag_dir_text, rag_dir_images, error_message, success = process_epub(temp_file_path, book_id)
    ...
    if success:
        folder_name = os.path.dirname(result_path)
        data = {
            "book_name": book_name,
            "book_id": book_id,
            "result_path": result_path,
            "folder_name": folder_name,
            "rag_dir_text": rag_dir_text,
            "rag_dir_images": rag_dir_images
        }

The core EPUB processing logic is in process_epub:

app/utils/utils.py (LNA app repo)
def process_epub(epub_file_path, book_id) -> Tuple[str, str, str,str, bool]:
    ...
    root_dir = os.getenv('ROOT_DIR')
    rag_dir = os.getenv('RAG_DIR')
    template_path = os.getenv('TEMPLATE_PATH')
    ...
    temp_folder = os.path.join(root_dir, book_id)
    ...
    # Unzip the epub
    files = unzip_epub(epub_file_path, temp_folder)
    ...
    # Choose ops_dir (OPS / OEBPS / EPUB)
    if len(matches) > 0:
        ops_dir = os.path.join(temp_folder, 'OPS')
    ...
    # Parse OPF
    if os.path.exists(os.path.join(ops_dir, 'content.opf')):
        with open(os.path.join(ops_dir, 'content.opf'), 'r', encoding='utf-8') as f:
            opf_content = f.read()
            title, nav_items, rag_dir_text, rag_dir_images = parse_opf_xml(opf_content, ops_dir, rag_dir)
    ...
    # Inject nav into viewer template
    viewer_html_path = os.path.join(ops_dir, 'viewer.html')
    success = add_nav_section(template_path, viewer_html_path, title, nav_items, f"{book_id}.mp3", book_id)
    ...
    return viewer_html_path, rag_dir_text, rag_dir_images, error_message, True

Audio pipeline

The audio endpoint (/audio/convert) converts .txt files from a directory into audio segments and concatenates them into a single {book_id}.mp3.

app/routers/audio.py (LNA app repo)
router = APIRouter(
    prefix="/audio",
    tags=["audio"],
    responses={404: {"description": "Not found"}},
)

@router.post("/convert", response_model=EpubProcResponse)
async def convert_audio(
    ops_dir: str = Form(...),
    destination_dir: str = Form(...),
    book_id: str = Form(...),
    current_user = Depends(get_current_active_user)
):
    ...
    txt_files = [f for f in os.listdir(source_dir) if f.endswith('.txt')]
    txt_files.sort(key=lambda x: int(os.path.splitext(os.path.basename(x))[0]) if os.path.splitext(os.path.basename(x))[0].isdigit() else float('inf'))
    ...
    book_audio_files, page = generate_audio(text, output_dir=destination_dir, page_number=page_number)
    ...
    allGood = combine_mp3_files(destination_dir, destination_dir + f'/{name}.mp3')
    ...
    return EpubProcResponse(success=True, message=f"Successfully converted {len(txt_files)} text files to audio")

Audio segment generation and concatenation are implemented here:

app/utils/audio_utils.py (LNA app repo)
def generate_audio(..., output_dir='output', page_number=0):
    pipeline = KPipeline(lang_code) 
    generator = pipeline(text, voice=voice, speed=speed, split_pattern=split_pattern)
    ...
    sf.write(f'{output_dir}/{page_number}.mp3', audio, 24000)
    fileList.append(f'{output_dir}/{page_number}.mp3')
    return fileList, page
app/utils/audio_utils.py (LNA app repo)
def combine_mp3_files(input_dir: str, output_file: str, ...):
    mp3_files = glob.glob(os.path.join(input_dir, file_pattern))
    ...
    mp3_files.sort(key=lambda x: int(os.path.splitext(os.path.basename(x))[0]) if os.path.splitext(os.path.basename(x))[0].isdigit() else float('inf'))
    ...
    final_audio = np.concatenate(combined_audio)
    sf.write(output_file, final_audio, sample_rate)
    return True

2. Start the service (port, run commands)

Default port

  • When running directly via python main.py, the server uses port 9000:
main.py (LNA app repo)
if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=9000)

Development scripts

  • start_server.sh starts the service on port 9001:
start_server.sh (LNA app repo)
source venv/bin/activate
uvicorn main:app --host 0.0.0.0 --reload --port 9001

Run steps

  1. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate
  1. Install dependencies
pip install -r requirements.txt

  1. Provide environment variables (see next section)

  2. Start the server (pick one):

# Option A (default port 9000)
uvicorn main:app --host 0.0.0.0 --port 9000 --reload

# Option B (script port 9001)
./start_server.sh

API docs

FastAPI exposes:

  • Swagger UI: http://localhost:<port>/docs
  • ReDoc: http://localhost:<port>/redoc

3. Configuration (.env)

The service expects a root .env at the project root.

Required variables used by EPUB processing and templating:

  • ROOT_DIR: where EPUB extraction temp folders are created (ROOT_DIR/<book_id>/...)
  • RAG_DIR: where extracted RAG text/images are stored
  • TEMPLATE_PATH: path to the HTML template used to generate viewer.html
  • API_URL: injected into the viewer template
  • BOOK_SOURCE_DIR: used by BookProcessor.py (optional background indexing)

The project also uses Supabase + Ollama for indexing/embeddings (used by app/utils/BookProcessor.py and app/utils/BookProcessor.py helpers).

You can also find a second .env under app/utils/.env (useful for running indexing scripts locally). Keep in mind that modules call load_dotenv() at import time, so the working directory matters.

4. Authentication (JWT)

Endpoints

  • POST /auth/login: returns a JWT access token
  • GET /auth/me: returns current user info
  • Protected endpoints require Authorization: Bearer <token>

Login endpoint is implemented as:

app/routers/auth.py (LNA app repo)
@router.post("/login", response_model=Token)
async def login_for_access_token(form_data: OAuth2PasswordRequestForm = Depends()):
    user = authenticate_user(form_data.username, form_data.password)
    ...
    access_token = create_access_token(
        data={"sub": user["username"]}, expires_delta=access_token_expires
    )
    return {"access_token": access_token, "token_type": "bearer"}

Default development user

The in-memory user database includes:

  • Username: testuser
  • Password: testpassword

5. HTTP API summary (what to call)

All non-auth endpoints require a valid JWT bearer token.

Root

  • GET / → { "message": "LNA BOOK Proc API is running" }

Health

  • GET /health

EPUB processing

  • POST /epub/proc
  • Content type: multipart/form-data
  • Form fields:
    • file: EPUB file
    • book_name: string
    • book_id: string

Validations:

  • File extension must end with .epub
  • File size must be <= 500MB
  • Rejects files with suspiciously small payload (< 100 bytes)

See:

app/routers/epub.py (LNA app repo)
if not file.filename.lower().endswith('.epub'):
    return EpubProcResponse(
        success=False,
        message=f"Invalid file type. Only EPUB files are allowed. {file.filename}"
    )
...
if file.size and file.size > MAX_FILE_SIZE:
    return EpubProcResponse(
        success=False,
        message=f"File too large. Maximum file size is {MAX_FILE_SIZE // (1024*1024)}MB. Your file is {file.size // (1024*1024)}MB."
    )
...
if len(content) < 100:
    return EpubProcResponse(
        success=False,
        message=f"File appears to be incomplete. Received {len(content)} bytes, which is too small for a valid EPUB file. Please check your upload."
    )

Response fields on success include:

  • result_path: generated viewer.html path
  • folder_name: directory that contains viewer.html
  • rag_dir_text: directory containing extracted numbered .txt chunks
  • rag_dir_images: directory containing images used by the viewer

Audio conversion

  • POST /audio/convert
  • Content type: multipart/form-data
  • Form fields:
    • ops_dir: directory containing numbered .txt files
    • destination_dir: directory to write {book_id}.mp3 into (usually alongside viewer.html)
    • book_id: controls output name {book_id}.mp3

6. EPUB → Viewer + RAG: end-to-end flow

Here is the intended flow based on the endpoint responses:

Client
  |
  | 1) POST /epub/proc (file, book_name, book_id) [JWT]
  v
API
  |
  | 2) unzip EPUB into ROOT_DIR/<book_id>/
  | 3) parse OPF (content.opf/contents.opf)
  | 4) write RAG text/images into RAG_DIR/<identifier>/{text,images}/
  | 5) generate ROOT_DIR/<book_id>/<OPS-or-OEBPS>/viewer.html
  |
  v
Client receives:
  - viewer.html path (result_path)
  - ops folder for assets (folder_name)
  - rag text folder (rag_dir_text)
  - rag images folder (rag_dir_images)
  |
  | (optional) 6) POST /audio/convert using rag_dir_text + folder_name [JWT]
  v
Audio output:
  - {folder_name}/{book_id}.mp3

Key implementation details to know when updating code:

  1. OPS folder detection: process_epub tries OPS, then OEBPS, then EPUB.
  2. OPF parsing: parse_opf_xml(...) scans the OPF manifest/spine and:
  3. creates RAG_DIR/<identifier>/text/
  4. creates RAG_DIR/<identifier>/images/
  5. writes numbered text chunks into .../text/<file_num>.txt
  6. Viewer templating: add_nav_section(...) modifies the HTML template:
  7. injects title + nav_items
  8. sets API URL and AI source book id
  9. attempts to set the audio file name in the viewer

The OPF parsing behavior (directory creation + text writing) lives here:

app/utils/epub_utils.py (LNA app repo)
def parse_opf_xml(html_data: str, ops_dir: str, rag_dir: str):
    ...
    identifier = soup.find('identifier')
    ...
    if len(spine_data) > 0:
        # create the folder for the RAG
        if not os.path.exists(os.path.join(rag_dir, identifier )):
            os.makedirs(os.path.join(rag_dir, identifier ))
        else:
            shutil.rmtree(os.path.join(rag_dir, identifier ))
            os.makedirs(os.path.join(rag_dir, identifier ))

        os.makedirs(os.path.join(rag_dir, identifier, 'text')) ...
        os.makedirs(os.path.join(rag_dir, identifier, 'images')) ...
        rag_dir_text = os.path.join(rag_dir, identifier, 'text')
        rag_dir_images = os.path.join(rag_dir, identifier, 'images')

Text chunk writing:

app/utils/epub_utils.py (LNA app repo)
if item['media-type'] == 'application/xhtml+xml':
    ...
    title, heading, data = get_text_from_xhtml(os.path.join(ops_dir,item['href'] ))
    if data is not None:
        try:
            file_name = f"{rag_dir}/{identifier}/text/{file_num}.txt"
            if "cover" in f"{item['id']}":
                file_name = f"{rag_dir}/{identifier}/text/0.txt"
            with open(file_name, 'w', encoding='utf-8') as f:
                f.write(data)
            file_num += 1
        ...
        nav_items.append({
            'id': item['id'],
            'index': index,
            'title': title,
            'subtitle': heading,
            'url': item['href']
        })

Viewer injection is done in html_util.py:

app/utils/html_util.py (LNA app repo)
def add_nav_section(template_path: str, html_path: str, section_title: str, nav_items: List[Dict[str, str]], audio_file_path: str, book_id: str) -> bool:
    ...
    api_url = soup.find('label', **{'id':'API_URL'})
    api_url.attrs['value'] = os.getenv('API_URL')
    ai_source = soup.find('label', **{'id':'AI_SOURCE'})
    ai_source.attrs['value'] = book_id
    ...
    aside = soup.find('aside', **{'class':'sidebar'})
    ...
    # Add the new nav_section to the sidebar
    aside.append(new_nav_section)
    ...
    with open(html_path, 'w', encoding='utf-8') as file:
        file.write(str(soup.prettify()))
    return True

7. Audio conversion: expected inputs

The audio endpoint sorts .txt files by their filename stem (numeric stems come first).

Implementation details:

app/routers/audio.py (LNA app repo)
txt_files = [f for f in os.listdir(source_dir) if f.endswith('.txt')]
txt_files.sort(key=lambda x: int(os.path.splitext(os.path.basename(x))[0]) if os.path.splitext(os.path.basename(x))[0].isdigit() else float('inf'))
...
with open(os.path.join(source_dir, file), 'r') as f:
    text = f.read()
book_audio_files, page = generate_audio(text, output_dir=destination_dir, page_number=page_number)
...
allGood = combine_mp3_files(destination_dir, destination_dir + f'/{name}.mp3')

Practical usage pattern:

  1. Call POST /epub/proc
  2. Use the response values:
  3. rag_dir_text → pass as ops_dir
  4. folder_name → pass as destination_dir
  5. book_id → pass as book_id

This causes the service to write:

  • {folder_name}/{book_id}.mp3

so the viewer can reference the correct audio file.

8. Optional: background indexing into Supabase (BookProcessor)

app/utils/BookProcessor.py is a standalone script that:

  • reads directories from BOOK_SOURCE_DIR
  • uses Ollama to:
  • generate titles/summaries (get_title_and_summary)
  • generate embeddings (get_embedding)
  • inserts processed chunks into Supabase table book_vector_pages

Key configuration:

app/utils/BookProcessor.py (LNA app repo)
load_dotenv()
llm_host = os.getenv('OLLAMA_HOST')
ollama_client = AsyncClient(host=llm_host)
...
def get_supabase_client() -> Client:
    supabase_url = os.getenv("SUPABASE_URL")
    supabase_key = os.getenv("SUPABASE_KEY")
    ...
    return create_client(supabase_url, supabase_key)

Embeddings + title/summary helpers:

app/utils/BookProcessor.py (LNA app repo)
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
    ...
async def get_embedding(text: str) -> List[float]:
    response = await ollama_client.embeddings(
        model="nomic-embed-text:latest",
        prompt=text
    )
    return response['embedding']

If you update RAG/chunking behavior, start here first.

9. Notes when updating code

  • Keep endpoint contracts stable:
  • /epub/proc request fields and response field names are relied upon by downstream consumers (audio conversion).
  • /audio/convert expects numbered .txt files for ordering.
  • Treat .env values as secrets:
  • Supabase keys and service keys must not be committed.
  • Be mindful of “working directory” effects:
  • several modules call load_dotenv() at import time.