Architecture

This page provides an overview of the MicroDC Python Client Library architecture. It is the current, maintained architecture reference. (An older, pre-refactor design document has been superseded and archived under docs/archive/DESIGN.md.)

High-Level Architecture

graph TB
    App["User Application"]

    subgraph lib["MicroDC Client Library"]
        Client["<b>client.py</b><br/>Authentication & Sessions<br/>Job Submission & Tracking<br/>File Upload & Tokens<br/>Callback Management"]

        subgraph jobs["jobs/"]
            BaseCall["base.py<br/><i>BaseCall</i>"]
            LLMComplete["llm_complete.py<br/><i>LLMComplete</i>"]
            LLMChat["llm_chat.py<br/><i>LLMChat</i>"]
            LLMEmbed["embed_call.py<br/><i>LLMEmbed</i>"]
            DocumentCall["document_call.py<br/><i>DocumentCall</i>"]
            ContainerJob["container_job.py<br/><i>ContainerJob</i>"]
            WorkerJob["worker_job.py<br/><i>WorkerJob</i>"]
            JobDetails["job_details.py<br/><i>JobDetails</i>"]
        end

        subgraph core["core/"]
            HTTP["http.py<br/><i>HTTPTransport</i>"]
            Polling["polling.py<br/><i>PollingManager</i>"]
            Encryption["encryption.py<br/><i>EncryptionManager</i>"]
            Config["config.py<br/><i>Config</i>"]
        end

        subgraph exceptions["exceptions/"]
            Errors["errors.py<br/><i>MicroDCError, APIError,<br/>AuthenticationError, ...</i>"]
        end
    end

    subgraph api["MicroDC Server REST API"]
        Submit["POST /api/v1/jobs/submit"]
        GetJob["GET /api/v1/jobs/{id}"]
        Ack["POST /api/v1/jobs/{id}/acknowledge"]
        Upload["POST /api/files/upload"]
    end

    App --> Client
    Client --> jobs
    Client --> core
    Client --> exceptions
    HTTP --> api

Module Responsibilities

Module	Purpose	Key Classes
client.py	Main API interface, job management	Client
jobs/base.py	Abstract job interface	BaseCall
jobs/llm_complete.py	Single-turn generation	LLMComplete
jobs/llm_chat.py	Multi-turn chat	LLMChat
jobs/embed_call.py	Embedding generation	LLMEmbed
jobs/document_call.py	Document processing	DocumentCall
jobs/container_job.py	Docker container execution	ContainerJob
jobs/worker_job.py	Folder-based streaming container	WorkerJob
jobs/job_details.py	Job result data model	JobDetails
core/http.py	HTTP transport layer	HTTPTransport
core/polling.py	Background job polling	PollingManager
core/encryption.py	Payload/result encryption	EncryptionManager
core/config.py	Configuration management	Config
exceptions/errors.py	Error handling	MicroDCError, etc.

Job Type Hierarchy

classDiagram
    BaseCall <|-- LLMComplete
    BaseCall <|-- LLMChat
    BaseCall <|-- LLMEmbed
    BaseCall <|-- DocumentCall
    BaseCall <|-- ContainerJob
    ContainerJob <|-- WorkerJob

    class BaseCall {
        <<abstract>>
        +metadata: Dict
        +priority: int
        +timeout: int
        +to_api_payload()* Dict
        +validate()*
    }
    class LLMComplete {
        +model: str
        +temperature: float
        +set_prompt(prompt)
    }
    class LLMChat {
        +model: str
        +temperature: float
        +set_system(content)
        +add_user_message(content)
        +add_assistant_message(content)
    }
    class LLMEmbed {
        +model: str
        +normalize: bool
        +add_text(text)
        +add_texts(texts)
    }
    class DocumentCall {
        +model: str
        +file_tokens: List
        +add_file(token)
        +add_files(tokens)
    }
    class ContainerJob {
        +image: str
        +set_image(image)
        +set_command(command)
        +enable_gpu()
        +add_env_var(key, value)
    }
    class WorkerJob {
        +folder_path: str
        +prepare() str
        +cleanup()
    }

Design Principles

1. Simplicity First -- Intuitive API for beginners, powerful for advanced users
2. Type Safety -- Full type hints for IDE support and static analysis
3. Async-First with Sync Support -- Callbacks for async, blocking waits for sync
4. Extensibility -- Base class design supports future job types
5. Fail-Safe Defaults -- Sensible defaults (temperature=0.7, auto polling)

API Endpoint Mapping

Client Method	HTTP	Endpoint
send_job(*)	POST	/api/v1/jobs/submit
get_job_details(id)	GET	/api/v1/jobs/{id}
get_job_status(id)	GET	/api/v1/jobs/{id}/status
cancel_job(id)	DELETE	/api/v1/jobs/{id}
acknowledge_job(id)	POST	/api/v1/jobs/{id}/acknowledge
list_jobs()	GET	/api/v1/jobs/
upload_file()	POST	/api/files/upload
create_download_token(id)	POST	/api/files/{id}/create-download-token

Polling Mechanism

The client uses a background thread for automatic job status polling:

1. Background thread starts on client initialization (configurable)
2. Polls every 2 seconds (configurable via poll_interval)
3. When a job completes, invokes the registered callback
4. Thread-safe for concurrent job submissions

Retry Strategy

Automatic retry for transient errors:

• Attempts: 3
• Base delay: 1 second
• Backoff: 2.0x exponential
• Retryable codes: 408, 429, 500, 502, 503, 504

HTTP Transport

Uses httpx.Client (HTTP/1.1) for:

• Thread-safe connection pooling
• Configurable timeouts
• Multipart file upload support
• Foundation for future async/await API

HTTP/2 is disabled

HTTP/2 was disabled in v1.0.13. httpcore's sync HTTP/2 multiplexes all concurrent requests onto one connection, and its per-stream bookkeeping is not thread-safe — sharing a Client across polling threads caused intermittent KeyError: <stream_id> crashes. The HTTP/1.1 pool is thread-safe and sufficient for these independent calls.