Integrate inference

Connect your engine using the Python or Rust provider SDK.

On this page

Connect payments to your engine

After onboarding, retain mppi.toml and its sidecar records. Add the Python or Rust composition functions below to your own inference host; they require your engine callbacks, listener and storage.

The provider SDK is a payment runtime. Your host supplies prepared token requirements, execution, accurate cumulative counts and definitive final outcomes. It also owns hosting, TLS, persistence and recovery infrastructure.

Stage / Your integration

Before admission: Measure input/cache requirements and an initial output bound without performing paid execution.
After admission: Dispatch once under the recorded reservation. Yield original JSON after consuming upstream framing.
More work: Call require_coverage with the complete desired remaining bound and proceed only if granted.
During execution: Report nondecreasing cumulative counts; do not double-count cached input.
Termination: Actually stop the job, then report definitive counts and COMPLETED, FAILED or CANCELLED.

Stage	Your integration
Before admission	Measure input/cache requirements and an initial output bound without performing paid execution.
After admission	Dispatch once under the recorded reservation. Yield original JSON after consuming upstream framing.
More work	Call `require_coverage` with the complete desired remaining bound and proceed only if granted.
During execution	Report nondecreasing cumulative counts; do not double-count cached input.
Termination	Actually stop the job, then report definitive counts and COMPLETED, FAILED or CANCELLED.

Python integration

Python · host composition fragment

import grpc
from mppi_provider import Mppi
from mppi_protocol.wire import GRPC_OPTIONS

async def attach_mppi(coverage, execution):
    sdk = await Mppi.from_config(
        "mppi.toml", testnet=True,
        coverage=coverage, execution=execution,
    )
    server = grpc.aio.server(options=GRPC_OPTIONS)
    sdk.register_grpc(server)
    readiness = await sdk.start()
    return sdk, server, readiness

Construct Mppi on the serving event loop. Register services before your server starts. from_config verifies saved configuration, the existing Circle operator and the trusted deployment; it starts no listener or inference job. The host keeps the Circle CLI and its original operation journal available.

Callback / Contract

await coverage(request, context): Return TokenCounts covering prepared prefill and at least the configured output buffer. request.payload contains validated original JSON bytes.
execution(request, context, runs, stop_requested): An async iterator of original JSON bytes. Use runs for coverage and reports. Observe stop_requested and terminate the real engine job.

Callback	Contract
`await coverage(request, context)`	Return `TokenCounts` covering prepared prefill and at least the configured output buffer. `request.payload` contains validated original JSON bytes.
`execution(request, context, runs, stop_requested)`	An async iterator of original JSON bytes. Use `runs` for coverage and reports. Observe `stop_requested` and terminate the real engine job.

Configure grpc.aio.server(options=GRPC_OPTIONS) to enforce transport limits. Expose all three discovery GET routes and both gRPC services at the declared origin. Return discovery status, headers and bytes unchanged.

Rust integration

Rust · composition fragment

use std::{path::Path, sync::Arc};
use mppi_provider::{
    cli::HostConfiguration,
    interfaces::{CoverageProvider, ExecutionProvider},
    runtime::Mppi,
    setup::{self, SetupError},
};

pub async fn compose(
    host: HostConfiguration,
    operation_record: &Path,
    coverage: Arc<dyn CoverageProvider>,
    execution: Arc<dyn ExecutionProvider>,
) -> Result<Arc<Mppi>, SetupError> {
    Ok(Arc::new(setup::runtime_from_config(
        host, operation_record, coverage, execution,
    ).await?))
}

Your executable obtains HostConfiguration from cli::host_configuration(std::env::args_os().skip(1)). Retain the validated listener settings before moving the configuration. Pass the original wallet-operation record, normally mppi.toml.wallet-operations.json.

Implement CoverageProvider and ExecutionProvider. Either register both services from ProviderLifecycle::grpc_services in your tonic host, or pass the runtime, ListenerConfig, shutdown CancellationToken and explicit ServePolicy to standalone::serve. The supplied listener serves discovery and both native gRPC services; your Tokio runtime needs I/O and timers.

runtime_from_config_with_options accepts RuntimeOptions for Circle options, polling, limits and hooks. Advanced hosts can construct Mppi::new with compatible wallet and chain capabilities. Your engine adapter, storage and job supervisor remain application code.

Shell · launch your compiled host

mppi provider serve --testnet --rpc-env ARC_RPC_URL_TESTNET \
  --host /absolute/path/to/your-provider-host

Report exactly what happened

Counts are cumulative per run. Cached tokens are a subset of input; positive cached counts require an accepted cache price. The SDK rejects decreasing counts/cost, work beyond the covered bound and conflicting definitive reports. A valid usage delta moves cost from reservation into usage once.

A cancellation event, network EOF or an exhausted iterator is not a final report. Retain reservations for unresolved work. Call report_terminal only with definitive counts and outcome after the engine establishes them; an exact repeated final is a no-op.

Separate output handling from payment progress. A stalled reader does not pause the engine. Respect bounded queues and terminate work when further coverage is denied; MPPI supplies no active-job pause/resume state.

Provider production checklist

Review the provider checklist