LLM providers¶
See the providers guide for installation and configuration.
LLMProvider ¶
Bases: ABC
Abstract base class for LLM providers.
context_window: int abstractmethod property ¶
Get the maximum context window size for this provider.
Returns:
| Type | Description |
|---|---|
int | Context window size in tokens |
model: str abstractmethod property ¶
Get the model name/identifier.
Returns:
| Type | Description |
|---|---|
str | Model name |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel abstractmethod ¶
Extract structured data from text using the LLM.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt to send to the LLM | required |
schema | type[BaseModel] | Pydantic model schema for the expected output | required |
temperature | float | Temperature for generation (0.0-1.0) | 0.0 |
max_tokens | int | None | Maximum tokens to generate | None |
Returns:
| Type | Description |
|---|---|
BaseModel | Instance of the schema with extracted data |
Raises:
| Type | Description |
|---|---|
LLMProviderError | If extraction fails |
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs: Any) -> str abstractmethod ¶
Get a text completion from the LLM.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt to complete | required |
temperature | float | Temperature for generation | 0.7 |
max_tokens | int | None | Maximum tokens to generate | None |
**kwargs | Any | Additional provider-specific arguments | {} |
Returns:
| Type | Description |
|---|---|
str | Completion text |
Raises:
| Type | Description |
|---|---|
LLMProviderError | If completion fails |
estimate_cost(text: str) -> float abstractmethod ¶
Estimate the API cost for processing this text.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text | str | Text to estimate cost for | required |
Returns:
| Type | Description |
|---|---|
float | Estimated cost in USD |
count_tokens(text: str) -> int abstractmethod ¶
Count tokens in text using provider's tokenizer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text | str | Text to count tokens for | required |
Returns:
| Type | Description |
|---|---|
int | Number of tokens |
supports_structured_output() -> bool ¶
Check if this provider supports structured output.
Returns:
| Type | Description |
|---|---|
bool | True if structured output is supported |
stream_complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs: Any) -> Iterator[str] ¶
Stream a text completion token-by-token.
The default implementation calls complete() and yields the full string as a single chunk. Providers with native streaming support (OpenAI, Anthropic) should override this to yield tokens as they arrive.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt to complete. | required |
temperature | float | Temperature for generation. | 0.7 |
max_tokens | int | None | Maximum tokens to generate. | None |
**kwargs | Any | Additional provider-specific arguments. | {} |
Yields:
| Type | Description |
|---|---|
str | str — successive token fragments of the completion. |
stream_complete_async(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs: Any) -> AsyncIterator[str] async ¶
Async version of stream_complete.
The default implementation wraps the synchronous generator. Providers with native async streaming should override this.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt to complete. | required |
temperature | float | Temperature for generation. | 0.7 |
max_tokens | int | None | Maximum tokens to generate. | None |
**kwargs | Any | Additional provider-specific arguments. | {} |
Yields:
| Type | Description |
|---|---|
AsyncIterator[str] | str — successive token fragments of the completion. |
LocalProvider ¶
Bases: LLMProvider
Local LLM provider using Ollama.
Supports running models locally for complete privacy and data sovereignty.
context_window: int property ¶
Get context window size.
model: str property ¶
Get model name.
__init__(model: str = 'llama-3.1-70b', host: str | None = None, temperature: float = 0.0, max_tokens: int = 4000) ¶
Initialize local LLM provider.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | str | Model name (must be pulled in Ollama) | 'llama-3.1-70b' |
host | str | None | Ollama host URL (defaults to OLLAMA_HOST env var or localhost) | None |
temperature | float | Default temperature for completions | 0.0 |
max_tokens | int | Default max tokens for completions | 4000 |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel ¶
Extract structured data using local LLM with JSON mode.
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) -> str ¶
Get text completion from local LLM.
estimate_cost(text: str) -> float ¶
Local LLMs have no API cost.
Returns:
| Type | Description |
|---|---|
float | Always returns 0.0 for local models |
count_tokens(text: str) -> int ¶
Count tokens using rough estimation.
Note: Ollama doesn't provide a token counting API.
OpenAIProvider ¶
Bases: LLMProvider
OpenAI LLM provider supporting GPT-4o, GPT-4o-mini, and other models.
context_window: int property ¶
Get context window size.
model: str property ¶
Get model name.
__init__(model: str = 'gpt-4o', api_key: str | None = None, temperature: float = 0.0, max_tokens: int = 4000) ¶
Initialize OpenAI provider.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | str | Model name (e.g., "gpt-4o", "gpt-4o-mini") | 'gpt-4o' |
api_key | str | None | OpenAI API key (defaults to OPENAI_API_KEY env var) | None |
temperature | float | Default temperature for completions | 0.0 |
max_tokens | int | Default max tokens for completions | 4000 |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel ¶
Extract structured data using OpenAI's structured output feature.
stream_complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) ¶
Stream a text completion from OpenAI token-by-token.
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) -> str ¶
Get text completion from OpenAI.
estimate_cost(text: str) -> float ¶
Estimate cost for processing text.
count_tokens(text: str) -> int ¶
Count tokens using tiktoken.
AnthropicProvider ¶
Bases: LLMProvider
Anthropic LLM provider supporting Claude 3.5 Sonnet and other models.
context_window: int property ¶
Get context window size.
model: str property ¶
Get model name.
__init__(model: str = 'claude-3-5-sonnet-20241022', api_key: str | None = None, temperature: float = 0.0, max_tokens: int = 4000) ¶
Initialize Anthropic provider.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | str | Model name (e.g., "claude-3-5-sonnet-20241022") | 'claude-3-5-sonnet-20241022' |
api_key | str | None | Anthropic API key (defaults to ANTHROPIC_API_KEY env var) | None |
temperature | float | Default temperature for completions | 0.0 |
max_tokens | int | Default max tokens for completions | 4000 |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel ¶
Extract structured data using Claude with JSON schema.
Claude doesn't have native structured output, so we inject the JSON schema into the prompt and parse the response.
stream_complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) ¶
Stream a text completion from Anthropic token-by-token.
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) -> str ¶
Get text completion from Anthropic.
estimate_cost(text: str) -> float ¶
Estimate cost for processing text.
count_tokens(text: str) -> int ¶
Count tokens using Anthropic's count method.
Falls back to character-based estimation if API unavailable.
GoogleProvider ¶
Bases: LLMProvider
Google Gemini LLM provider supporting gemini-pro and other models.
model: str property ¶
Get the model name.
context_window: int property ¶
Get the context window size for the model.
__init__(model: str = 'models/gemini-2.0-flash', api_key: str | None = None, temperature: float = 0.0, max_tokens: int = 4000) ¶
Initialize Google Gemini provider.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | str | Model name (e.g., "models/gemini-2.0-flash", "models/gemini-2.5-pro") | 'models/gemini-2.0-flash' |
api_key | str | None | Google API key (defaults to GOOGLE_API_KEY env var) | None |
temperature | float | Default temperature for completions | 0.0 |
max_tokens | int | Default max tokens for completions | 4000 |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel ¶
Extract structured data using Google Gemini.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt describing what to extract | required |
schema | type[BaseModel] | Pydantic model defining the expected structure | required |
temperature | float | Sampling temperature (0.0 = deterministic) | 0.0 |
max_tokens | int | None | Maximum tokens to generate | None |
Returns:
| Type | Description |
|---|---|
BaseModel | Instance of schema with extracted data |
Raises:
| Type | Description |
|---|---|
LLMProviderError | If extraction fails |
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs: Any) -> str ¶
Get text completion from Google Gemini.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prompt | str | The prompt text | required |
temperature | float | Sampling temperature | 0.7 |
max_tokens | int | None | Maximum tokens to generate | None |
**kwargs | Any | Additional arguments | {} |
Returns:
| Type | Description |
|---|---|
str | Generated text |
count_tokens(text: str) -> int ¶
Count tokens in text.
Note: Google doesn't provide a simple token counter in the SDK, so we use a rough approximation.
estimate_cost(text: str) -> float ¶
Estimate cost for processing text.
LangChainProvider ¶
Bases: LLMProvider
Adapter to use LangChain LLMs with Contractex.
This allows using any LangChain-compatible LLM with Contractex.
context_window: int property ¶
Get context window size.
model: str property ¶
Get model name.
__init__(langchain_llm: Any, default_max_tokens: int = 4000) ¶
Initialize LangChain provider adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
langchain_llm | Any | Any LangChain LLM instance | required |
default_max_tokens | int | Default max tokens for completions | 4000 |
extract_structured(prompt: str, schema: type[BaseModel], temperature: float = 0.0, max_tokens: int | None = None) -> BaseModel ¶
Extract structured data using LangChain LLM.
Uses LangChain's structured output if available, otherwise falls back to JSON parsing.
complete(prompt: str, temperature: float = 0.7, max_tokens: int | None = None, **kwargs) -> str ¶
Get text completion from LangChain LLM.
estimate_cost(text: str) -> float ¶
Estimate cost for processing text.
Returns 0.0 if LangChain LLM doesn't provide pricing info.
count_tokens(text: str) -> int ¶
Count tokens using LangChain's method if available.