Foundry Local SDK 参考

Python SDK 参考

安装

安装 Python 包：

pip install foundry-local-sdk

FoundryLocalManager 类

该 FoundryLocalManager 类提供用于管理模型、缓存和 Foundry Local 服务的方法。

初始化

from foundry_local import FoundryLocalManager

# Initialize and optionally bootstrap with a model
manager = FoundryLocalManager(alias_or_model_id=None, bootstrap=True)

• alias_or_model_id：（可选）在启动时下载和加载的别名或模型 ID。
• bootstrap：（默认值 True） 如果为 True，则启动服务（如果未运行），并在提供时加载模型。

有关别名的注释

此引用中概述的许多方法在签名中都有一个 alias_or_model_id 参数。 可以将 别名 或 模型 ID 作为值传入方法。 使用别名将：

• 选择适合可用硬件的最佳模型。 例如，如果 Nvidia CUDA GPU 可用，Foundry Local 将选择 CUDA 模型。 如果支持的 NPU 可用，Foundry Local 将选择 NPU 模型。
• 允许使用较短的名称，而无需记住模型 ID。

服务管理

目录管理

缓存管理

模型管理

示例用法

以下代码演示如何使用 FoundryManager 类管理模型并与 Foundry Local 服务交互。

from foundry_local import FoundryLocalManager

# By using an alias, the most suitable model will be selected 
# to your end-user's device.
alias = "phi-3.5-mini"

# Create a FoundryLocalManager instance. This will start the Foundry.
manager = FoundryLocalManager()

# List available models in the catalog
catalog = manager.list_catalog_models()
print(f"Available models in the catalog: {catalog}")

# Download and load a model
model_info = manager.download_model(alias)
model_info = manager.load_model(alias)
print(f"Model info: {model_info}")

# List models in cache
local_models = manager.list_cached_models()
print(f"Models in cache: {local_models}")

# List loaded models
loaded = manager.list_loaded_models()
print(f"Models running in the service: {loaded}")

# Unload a model
manager.unload_model(alias)

与 OpenAI SDK 集成

安装 OpenAI 包：

pip install openai

以下代码演示如何使用 OpenAI SDK 集成 FoundryLocalManager 以与本地模型交互。

import openai
from foundry_local import FoundryLocalManager

# By using an alias, the most suitable model will be downloaded 
# to your end-user's device.
alias = "phi-3.5-mini"

# Create a FoundryLocalManager instance. This will start the Foundry 
# Local service if it is not already running and load the specified model.
manager = FoundryLocalManager(alias)

# The remaining code us es the OpenAI Python SDK to interact with the local model.

# Configure the client to use the local Foundry service
client = openai.OpenAI(
    base_url=manager.endpoint,
    api_key=manager.api_key  # API key is not required for local usage
)

# Set the model to use and generate a streaming response
stream = client.chat.completions.create(
    model=manager.get_model_info(alias).id,
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
    stream=True
)

# Print the streaming response
for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)

JavaScript SDK 参考文档

安装

请从 npm 安装该软件包：

npm install foundry-local-sdk

FoundryLocalManager 类

该 FoundryLocalManager 类允许你在浏览器和 Node.js 环境中管理模型、控制缓存以及与 Foundry Local 服务交互。

初始化

import { FoundryLocalManager } from "foundry-local-sdk";

const foundryLocalManager = new FoundryLocalManager()

可用选项：

• serviceUrl：Foundry Local 服务的基 URL
• fetch：（可选） 适用于 Node.js 等环境的自定义提取实现

有关别名的注释

此引用中概述的许多方法在签名中都有一个 aliasOrModelId 参数。 可以将 别名 或 模型 ID 作为值传入方法。 使用别名将：

• 选择适合可用硬件的最佳模型。 例如，如果 Nvidia CUDA GPU 可用，Foundry Local 将选择 CUDA 模型。 如果支持的 NPU 可用，Foundry Local 将选择 NPU 模型。
• 允许使用较短的名称，而无需记住模型 ID。

服务管理

目录管理

缓存管理

模型管理

示例用法

以下代码演示如何使用 FoundryLocalManager 类管理模型并与 Foundry Local 服务交互。

import { FoundryLocalManager } from "foundry-local-sdk";

// By using an alias, the most suitable model will be downloaded 
// to your end-user's device.
// TIP: You can find a list of available models by running the 
// following command in your terminal: `foundry model list`.
const alias = "phi-3.5-mini";

const manager = new FoundryLocalManager()

// Initialize the SDK and optionally load a model
const modelInfo = await manager.init(alias)
console.log("Model Info:", modelInfo)

// Check if the service is running
const isRunning = await manager.isServiceRunning()
console.log(`Service running: ${isRunning}`)

// List available models in the catalog
const catalog = await manager.listCatalogModels()

// Download and load a model
await manager.downloadModel(alias)
await manager.loadModel(alias)

// List models in cache
const localModels = await manager.listCachedModels()

// List loaded models
const loaded = await manager.listLoadedModels()

// Unload a model
await manager.unloadModel(alias)

与 OpenAI 客户端集成

安装 OpenAI 包：

npm install openai

以下代码演示如何将 FoundryLocalManager 与 OpenAI 客户端集成，以便与本地模型进行交互。

import { OpenAI } from "openai";
import { FoundryLocalManager } from "foundry-local-sdk";

// By using an alias, the most suitable model will be downloaded 
// to your end-user's device.
// TIP: You can find a list of available models by running the 
// following command in your terminal: `foundry model list`.
const alias = "phi-3.5-mini";

// Create a FoundryLocalManager instance. This will start the Foundry 
// Local service if it is not already running.
const foundryLocalManager = new FoundryLocalManager()

// Initialize the manager with a model. This will download the model 
// if it is not already present on the user's device.
const modelInfo = await foundryLocalManager.init(alias)
console.log("Model Info:", modelInfo)

const openai = new OpenAI({
  baseURL: foundryLocalManager.endpoint,
  apiKey: foundryLocalManager.apiKey,
});

async function streamCompletion() {
    const stream = await openai.chat.completions.create({
      model: modelInfo.id,
      messages: [{ role: "user", content: "What is the golden ratio?" }],
      stream: true,
    });
  
    for await (const chunk of stream) {
      if (chunk.choices[0]?.delta?.content) {
        process.stdout.write(chunk.choices[0].delta.content);
      }
    }
}
  
streamCompletion();

浏览器使用情况

SDK 包含与浏览器兼容的版本，必须在其中手动指定服务 URL：

import { FoundryLocalManager } from "foundry-local-sdk/browser"

// Specify the service URL
// Run the Foundry Local service using the CLI: `foundry service start`
// and use the URL from the CLI output
const endpoint = "ENDPOINT"

const manager = new FoundryLocalManager({serviceUrl: endpoint})

// Note: The `init`, `isServiceRunning`, and `startService` methods 
// are not available in the browser version

示例用法

import { FoundryLocalManager } from "foundry-local-sdk/browser"

// Specify the service URL
// Run the Foundry Local service using the CLI: `foundry service start`
// and use the URL from the CLI output
const endpoint = "ENDPOINT"

const manager = new FoundryLocalManager({serviceUrl: endpoint})

const alias = 'phi-3.5-mini'

// Get all available models
const catalog = await manager.listCatalogModels()
console.log("Available models in catalog:", catalog)

// Download and load a specific model
await manager.downloadModel(alias)
await manager.loadModel(alias)

// View models in your local cache
const localModels = await manager.listLocalModels()
console.log("Cached models:", catalog)

// Check which models are currently loaded
const loaded = await manager.listLoadedModels()
console.log("Loaded models in inference service:", loaded)

// Unload a model when finished
await manager.unloadModel(alias)