Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions sdk_v2/cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -478,10 +478,12 @@ if(FOUNDRY_LOCAL_BUILD_EXAMPLES)
add_subdirectory(examples/tool_calling)
add_subdirectory(examples/realtime_audio)
add_subdirectory(examples/embeddings)
add_subdirectory(examples/catalog)
set_target_properties(basic_chat_example PROPERTIES FOLDER "Examples")
set_target_properties(tool_calling_example PROPERTIES FOLDER "Examples")
set_target_properties(realtime_audio_example PROPERTIES FOLDER "Examples")
set_target_properties(embeddings_example PROPERTIES FOLDER "Examples")
set_target_properties(catalog_example PROPERTIES FOLDER "Examples")

# On Linux, GenAI does dlopen("libonnxruntime.so") internally. Modern GCC
# emits RUNPATH which doesn't propagate to transitive dlopen calls.
Expand All @@ -491,5 +493,6 @@ if(FOUNDRY_LOCAL_BUILD_EXAMPLES)
target_link_options(tool_calling_example PRIVATE -Wl,--disable-new-dtags)
target_link_options(realtime_audio_example PRIVATE -Wl,--disable-new-dtags)
target_link_options(embeddings_example PRIVATE -Wl,--disable-new-dtags)
target_link_options(catalog_example PRIVATE -Wl,--disable-new-dtags)
endif()
endif()
19 changes: 17 additions & 2 deletions sdk_v2/cpp/docs/CppPortGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ FoundryLocalCore (singleton, DI container)

```
Manager (singleton, explicit Create/Destroy lifecycle)
├── ICatalog (BaseModelCatalog → AzureModelCatalog, LocalModelScanner)
├── ICatalog (BaseModelCatalog → AzureModelCatalog, LocalModelScanner); N named, addressed individually
├── ModelLoadManager (mutex-guarded map<id, unique_ptr<GenAIModelInstance>>)
├── Session/ChatSession/AudioSession (stateful, owns conversation history)
├── IEpDetector (EpDetector — real detection + CUDA bootstrapping)
Expand Down Expand Up @@ -184,7 +184,7 @@ Both are move-only / non-copyable. C# uses `IDisposable`; C++ uses RAII via `uni
|----|-----|-------|
| `IModelCatalog<T>` generic interface | `ICatalog` non-generic interface | C++ drops the generic; all catalogs produce `Model` |
| `BaseModelCatalog<T>` | `BaseModelCatalog` | Same role: lazy population, indexed lookup |
| `AggregateModelCatalog<T>` | *(not ported)* | C++ uses a single catalog with multiple sources internally |
| `AggregateModelCatalog<T>` | *(not ported)* | C++ holds N separately addressable catalogs instead of merging them |
| `CachedInfo` struct | `ModelIndex` (atomic `shared_ptr`) | C++ uses lock-free index swap for concurrent reads |
| `AsyncLock` | `std::mutex` + `std::lock_guard` | Different concurrency primitives |

Expand All @@ -193,6 +193,21 @@ C++ catalog uses **three indices** (by id, by alias, by name) stored in an
gives lock-free reads during catalog queries. The C# version uses `AsyncLock` around
reads/writes.

**Separately addressable catalogs.** Rather than merging every source into one aggregate,
the Manager holds N catalogs, each registered under a unique name and each backed by its own
`AzureModelCatalog`. Results are never unioned or de-duplicated across catalogs — you address
one catalog at a time:

- Configure sources with `Configuration::AddCatalog(name, url, filter?)`. `AddCatalogUrl(url)`
remains supported and registers the catalog under an auto-derived name (its URL).
- `Manager::GetCatalog(name)` returns the named catalog; `Manager::ListCatalogNames()`
enumerates registered names in add-order.
- The no-argument `Manager::GetCatalog()` returns the first-registered catalog (the default).
When no source is added, the built-in Azure Foundry catalog is registered under the reserved
name `"public"`, which then serves as the default.
- The C ABI exposes this via `Manager_GetCatalogByName` and `Manager_ListCatalogNames`; an
unknown name yields `FOUNDRY_LOCAL_ERROR_INVALID_ARGUMENT`.

The C++ `Model` class has two modes:
- **Leaf:** Single model variant with its own `ModelInfo`
- **Container:** Multi-variant wrapper that delegates property access to a selected variant
Expand Down
3 changes: 3 additions & 0 deletions sdk_v2/cpp/docs/WrapperInterfacesDesign.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,9 @@ Driven by the interface decisions:
ctor for the rare case where a user holds a raw `flCatalog*` from the C API and
wants a wrapper directly. There is no `friend` relationship between `Manager` and
`Catalog` — the public ctor is consistent with every other top-level wrapper.
`Manager::GetCatalog(name)` returns a specific named catalog and `Manager::ListCatalogNames()`
enumerates the registered names; the no-argument overload returns the first-registered
(default) catalog.
- `Catalog::GetModel(alias)` and `Catalog::GetModelVariant(id)` return
`std::unique_ptr<IModel>`. Null = not found.
- `Catalog::GetLatestVersion(const IModel&)` returns `std::unique_ptr<IModel>`.
Expand Down
4 changes: 4 additions & 0 deletions sdk_v2/cpp/examples/catalog/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Copyright (c) Microsoft. All rights reserved.

add_executable(catalog_example main.cc)
target_link_libraries(catalog_example PRIVATE foundry_local_cpp)
153 changes: 153 additions & 0 deletions sdk_v2/cpp/examples/catalog/main.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
//
// Example: Interactive exploration of multiple separately addressable catalogs.
//
// This is a small REPL. You register named catalogs, enumerate them, select one,
// and query its models — demonstrating that each catalog is addressed and served
// independently (no aggregation across catalogs).
//
// Catalogs are configured up front (before the Manager is created), so the REPL
// rebuilds the Manager whenever you add a catalog. The `models` command performs
// a live network query against the selected catalog's URL.

#include <foundry_local/foundry_local_cpp.h>

#include <iostream>
#include <memory>
#include <sstream>
#include <string>
#include <vector>

using namespace foundry_local;

namespace {

// A registered catalog source.
struct Source {
std::string name;
std::string url;
};

void PrintHelp() {
std::cout <<
"\nCommands:\n"
" list List registered catalog names\n"
" add <name> <url> Register a named catalog (rebuilds the manager)\n"
" use <name> Select a catalog as the current one\n"
" name Show the current catalog's reported name\n"
" models List models from the current catalog (live query)\n"
" help Show this help\n"
" quit Exit\n\n";
}

// Build a Manager from the registered sources. With no sources, the built-in
// "public" catalog is the default.
std::unique_ptr<Manager> BuildManager(const std::vector<Source>& sources) {
Configuration config("catalog_repl");
for (const auto& s : sources) {
config.AddCatalog(s.name, s.url);
}
return std::make_unique<Manager>(std::move(config));
}

void ListCatalogs(const Manager& manager) {
std::vector<std::string> names = manager.ListCatalogNames();
std::cout << "Registered catalogs (" << names.size() << "):\n";
for (const auto& name : names) {
std::cout << " - " << name << "\n";
}
}

void ListModels(Manager& manager, const std::string& current) {
try {
ICatalog& catalog = current.empty() ? manager.GetCatalog() : manager.GetCatalog(current);
std::cout << "Querying '" << (current.empty() ? std::string("<default>") : current)
<< "'...\n";
ModelList models = catalog.GetModels();
std::cout << "Models (" << models.size() << "):\n";
for (const auto& model : models.Models()) {
ModelInfo info = model->GetInfo();
std::cout << " - " << info.Alias() << " (" << info.Id() << ")\n";
}
} catch (const Error& ex) {
std::cout << "Query failed: " << ex.what() << "\n";
}
}

} // namespace

int main() {
std::vector<Source> sources;
std::unique_ptr<Manager> manager = BuildManager(sources);
std::string current; // empty = default catalog

std::cout << "Interactive catalog demo. Type 'help' for commands.\n";
ListCatalogs(*manager);

std::string line;
while (true) {
std::cout << "\n[" << (current.empty() ? "default" : current) << "] > " << std::flush;
if (!std::getline(std::cin, line)) {
break; // EOF
}

std::istringstream iss(line);
std::string cmd;
iss >> cmd;

if (cmd.empty()) {
continue;
} else if (cmd == "quit" || cmd == "exit") {
break;
} else if (cmd == "help") {
PrintHelp();
} else if (cmd == "list") {
ListCatalogs(*manager);
} else if (cmd == "add") {
std::string name, url;
iss >> name >> url;
if (name.empty() || url.empty()) {
std::cout << "Usage: add <name> <url>\n";
continue;
}
try {
sources.push_back({name, url});
manager.reset(); // Manager is a singleton — destroy the old one first.
manager = BuildManager(sources);
current.clear();
std::cout << "Added '" << name << "'. Manager rebuilt.\n";
ListCatalogs(*manager);
} catch (const Error& ex) {
sources.pop_back();
manager.reset();
manager = BuildManager(sources);
std::cout << "Failed to add catalog: " << ex.what() << "\n";
}
} else if (cmd == "use") {
std::string name;
iss >> name;
if (name.empty()) {
std::cout << "Usage: use <name>\n";
continue;
}
try {
(void)manager->GetCatalog(name); // validate the name exists
current = name;
std::cout << "Current catalog: " << current << "\n";
} catch (const Error& ex) {
std::cout << "Unknown catalog: " << ex.what() << "\n";
}
} else if (cmd == "name") {
ICatalog& catalog = current.empty() ? manager->GetCatalog() : manager->GetCatalog(current);
std::cout << "Reported name: " << catalog.GetName() << "\n";
} else if (cmd == "models") {
ListModels(*manager, current);
} else {
std::cout << "Unknown command: " << cmd << " (type 'help')\n";
}
}

std::cout << "Bye.\n";
return 0;
}
25 changes: 25 additions & 0 deletions sdk_v2/cpp/include/foundry_local/foundry_local_c.h
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,8 @@ typedef struct flApi {
FL_API_STATUS(Manager_Create, _In_ const flConfiguration* config, _Outptr_ flManager** out_manager);
FL_TYPE_RELEASE(Manager);

/// Get the default catalog: the first registered catalog, or the built-in "public"
/// catalog when none was added. For a specific catalog, use Manager_GetCatalogByName.
FL_API_STATUS(Manager_GetCatalog, _In_ const flManager* manager, _Outptr_ flCatalog** out_catalog);
FL_API_STATUS(Manager_WebServiceStart, _In_ flManager* manager);
// Get the bound service urls. Returns success with *out_num_urls == 0 when the web service is not running;
Expand Down Expand Up @@ -705,6 +707,19 @@ typedef struct flApi {
/// Check if Shutdown has been called.
bool FL_API_T(Manager_IsShutdownRequested, _In_ const flManager* manager);

/// Get a registered catalog by name. Returns FOUNDRY_LOCAL_ERROR_INVALID_ARGUMENT
/// if no catalog with that name is registered. The returned flCatalog is owned by
/// the Manager and remains valid for its lifetime.
FL_API_STATUS(Manager_GetCatalogByName, _In_ const flManager* manager, _In_ const char* name,
_Outptr_ flCatalog** out_catalog);

/// Enumerate the names of all registered catalogs, in registration order. The first
/// name is the default catalog returned by Manager_GetCatalog. The returned array and
/// its strings are owned by the Manager and remain valid until the next call to
/// Manager_ListCatalogNames on the same manager or until the Manager is released.
FL_API_STATUS(Manager_ListCatalogNames, _In_ const flManager* manager,
_Outptr_result_buffer_(*out_count) const char* const** out_names, _Out_ size_t* out_count);

// End V1
/* Append new function pointers at the end for future versions and add marker for the end of each version */
} flApi;
Expand Down Expand Up @@ -908,6 +923,8 @@ struct flConfigurationApi {
FL_API_STATUS(SetModelCacheDir, _In_ flConfiguration* config, _In_ const char* dir);
/// Optional. Add a catalog URL. Defaults to the Azure Foundry Local Catalog if none added.
/// Multiple catalogs can be added. Catalogs priority is determined by the order they were added.
/// The catalog is registered under an auto-derived name (its URL); to give it an explicit
/// name for scoped operations, use AddCatalog instead.
/// @param filter_override Optional filter string for this catalog. Pass NULL for no override.
FL_API_STATUS(AddCatalogUrl, _In_ flConfiguration* config, _In_ const char* url,
_In_opt_ const char* filter_override);
Expand All @@ -929,6 +946,14 @@ struct flConfigurationApi {
/// These are passed through to the core implementation. The configuration copies the data.
FL_API_STATUS(SetAdditionalOptions, _In_ flConfiguration* config, _In_ const flKeyValuePairs* options);

/// Optional. Add a named catalog. Defaults to the Azure Foundry Local Catalog if none added.
/// Multiple catalogs can be added; each must have a unique name. The first added catalog is the default.
/// Each catalog is addressed independently by name for scoped list/download operations.
/// The name "public" is reserved for the built-in default catalog.
/// @param filter_override Optional filter string for this catalog. Pass NULL for no override.
FL_API_STATUS(AddCatalog, _In_ flConfiguration* config, _In_ const char* name, _In_ const char* url,
_In_opt_ const char* filter_override);

// End V1
};

Expand Down
30 changes: 27 additions & 3 deletions sdk_v2/cpp/include/foundry_local/foundry_local_cpp.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#include <cstdlib>
#include <functional>
#include <gsl/span>
#include <map>
#include <memory>
#include <mutex>
#include <optional>
Expand Down Expand Up @@ -249,9 +250,18 @@ class Configuration {

/// Optional. Add a catalog URL to connect to.
/// Defaults to the Azure Foundry Local Catalog if none are added.
/// The catalog is registered under an auto-derived name (its URL); use AddCatalog to
/// assign an explicit name for scoped list/download operations.
Configuration& AddCatalogUrl(const std::string& url,
const std::optional<std::string>& filter_override = std::nullopt);

/// Optional. Add a named catalog to connect to.
/// Defaults to the Azure Foundry Local Catalog if none are added.
/// Each catalog must have a unique name; the name is used to address the catalog for
/// scoped operations. The name "public" is reserved for the built-in default catalog.
Configuration& AddCatalog(const std::string& name, const std::string& url,
const std::optional<std::string>& filter_override = std::nullopt);

/// Optional. Add an endpoint for the web service to bind to.
/// Defaults to "http://127.0.0.1:0" (ephemeral port) if none are added.
Configuration& AddWebServiceEndpoint(const std::string& url);
Expand Down Expand Up @@ -829,9 +839,18 @@ class Manager {

const Configuration& GetConfiguration() const { return config_; }

/// Get the catalog for querying models. Creates on first call, caches internally.
/// Get the default catalog for querying models. This is the first registered catalog, or the
/// built-in "public" catalog when none was added. Creates on first call, caches internally.
ICatalog& GetCatalog() const;

/// Get a registered catalog by name. Throws if no catalog with that name is registered.
/// The returned catalog is cached, so repeated calls with the same name yield the same object.
ICatalog& GetCatalog(const std::string& name) const;

/// Enumerate the names of all registered catalogs, in registration order. The first name
/// corresponds to the default catalog returned by the no-argument GetCatalog().
std::vector<std::string> ListCatalogNames() const;

/// Start the embedded web service.
void StartWebService();

Expand Down Expand Up @@ -865,8 +884,13 @@ class Manager {
private:
detail::Base<flManager> handle_;
Configuration config_;
mutable std::unique_ptr<Catalog> catalog_;
mutable std::unique_ptr<std::once_flag> catalog_once_{std::make_unique<std::once_flag>()};
// Cache of named catalog wrappers, guarded by named_catalogs_mutex_ for concurrent access.
// The no-argument GetCatalog() resolves the default catalog through this same cache, so both
// access paths return one canonical wrapper per catalog.
mutable std::map<std::string, std::unique_ptr<Catalog>> named_catalogs_;
mutable std::unique_ptr<std::mutex> named_catalogs_mutex_{std::make_unique<std::mutex>()};
mutable std::string default_catalog_name_;
mutable std::unique_ptr<std::once_flag> default_catalog_once_{std::make_unique<std::once_flag>()};
};

// ===========================================================================
Expand Down
Loading
Loading