⚙️ Configuration Architecture & Precedence¶

Spector's unified configuration subsystem powers everything from microsecond embedded vector search to multi-tenant distributed cognitive memory clusters. This guide outlines the 6-layer configuration hierarchy, profile activation, discovery paths, and runtime resolution mechanics.

🏛️ Configuration Architecture¶

Spector employs a layered, hierarchical configuration engine built in nucleus/spector-config centered on SpectorConfigSource. Under the hood, it layers multiple configuration sources using Apache Commons Configuration 2 with an OverrideCombiner.

Higher layers take precedence over lower layers, allowing global defaults to be established in version control while environment-specific secrets, container volumes, and runtime flags selectively override individual keys.

flowchart TD
    subgraph Resolution["Resolution Hierarchy (Highest Priority Wins)"]
        direction TB
        L1["1. Programmatic Overrides<br/><code>SpectorConfig.DEFAULT.with...</code>"]
        L2["2. Java System Properties<br/><code>-Dspector.provider.embedding.dimensions=768</code>"]
        L3["3. Environment Variables<br/><code>SPECTOR_PROVIDER_EMBEDDING_DIMENSIONS=768</code>"]
        L4["4. Profile Configuration<br/><code>spector-{profile}.yml</code>"]
        L5["5. Local YAML<br/><code>spector.yml (Working Directory)</code>"]
        L6["6. Bundled Classpath Defaults<br/><code>spector-defaults.yml (JAR)</code>"]
    end

    L1 --> L2
    L2 --> L3
    L3 --> L4
    L4 --> L5
    L5 --> L6

    subgraph Consumer["Spector Runtime Components"]
        Engine["SpectorMemory Kernel"]
        Index["SpectorIndex HNSW / Quant"]
        Synapse["Spector Synapse REST & MCP"]
        CLI["Spector CLI & Ingestion"]
    end

    Resolution --> Consumer

🪜 The 6-Layer Resolution Order¶

Layer 1: Programmatic Overrides¶

When using Spector embedded as a Java library, code-level builder invocations override any external configuration files:

import com.spectrayan.spector.config.SpectorConfig;
import com.spectrayan.spector.model.QuantizationType;

// Programmatic configuration completely overrides environment and YAML defaults
var config = SpectorConfig.DEFAULT
    .withDimensions(768)
    .withCapacity(1_000_000)
    .withQuantization(QuantizationType.SCALAR_INT4)
    .withRescore(3);

Layer 2: Java System Properties¶

System properties passed on the command line via -D override environment variables and configuration files:

java \
  --enable-preview --add-modules=jdk.incubator.vector --enable-native-access=ALL-UNNAMED \
  -Dspector.provider.embedding.dimensions=1536 \
  -Dspector.provider.embedding.type=openai \
  -jar spector-synapse.jar

Layer 3: Environment Variables & Secrets¶

Every dot-notation configuration property automatically maps to a screaming snake_case environment variable. Dots and hyphens are replaced with underscores, and the key is uppercased:

• spector.provider.embedding.dimensions \(\to\) SPECTOR_PROVIDER_EMBEDDING_DIMENSIONS
• spector.hnsw.ef-construction \(\to\) SPECTOR_HNSW_EF_CONSTRUCTION
• spector.provider.embedding.api-key \(\to\) SPECTOR_PROVIDER_EMBEDDING_API_KEY

For container convenience, Spector also supports short aliases (e.g. SPECTOR_EMBEDDING_PROVIDER) and Docker secrets (/run/secrets/). See the Environment Variables & Secrets Guide for the complete mapping table.

Layer 4: Profile Configuration (spector-{profile}.yml)¶

Profiles allow you to package environment-specific configurations cleanly. When a profile is active, Spector loads spector-{profile}.yml and overlays it on top of spector.yml.

Activating a Profile¶

Profiles can be activated using either an environment variable or a command-line argument:

# Via Environment Variable
export SPECTOR_PROFILE=docker

# Via Spring / Synapse CLI argument
java -jar spector-synapse.jar --spector.profile=production

Common Profiles Bundled with Spector¶

• spector-docker.yml — Optimized for containerized environments (/data/memory persistence, host.docker.internal networking, reduced memory footprint).
• spector-bench.yml — Tuned for maximum recall benchmarks (uncompressed vectors, elevated \(efSearch\), disabled background sleep consolidation).
• spector-test.yml — In-memory persistence (MEMORY), zero disk footprint, ephemeral test isolation.

Layer 5: Local spector.yml¶

A spector.yml file placed in the application's working directory provides baseline project configuration. If present, it overrides the bundled defaults:

spector:
  mode: MEMORY
  memory:
    persistence-mode: DISK
    persistence-path: ./data/memory
    dimensions: 768
  provider:
    embedding:
      type: ollama
      model: nomic-embed-text

Layer 6: Classpath Defaults (spector-defaults.yml)¶

The JAR bundles spector-defaults.yml (located in nucleus/spector-config/src/main/resources/spector-defaults.yml). This file guarantees that Spector always has valid, production-hardened defaults for all 35 property groups even if no configuration files or environment variables are provided.

🔍 Discovery & File Search Paths¶

When bootstrapping via SpectorConfigSource.load(), Spector searches for YAML configuration files in the following sequence:

1. Explicit Path: If specified via SpectorConfigSource.load(Path.of("/custom/path.yml")) or --spring.config.additional-location=file:/app/spector.yml.
2. Current Working Directory: ./spector-{profile}.yml then ./spector.yml.
3. User Home Directory: ~/.spector/spector-{profile}.yml then ~/.spector/spector.yml.
4. System Configuration Directory: /etc/spector/spector.yml (on Linux / Unix systems).
5. Classpath: classpath:spector-defaults.yml.

🛡️ Validation & Error Handling¶

Configuration loading performs strict validation on initialization. If an invalid type, unparseable duration, or out-of-range value is encountered, Spector fails fast with clear diagnostic exceptions:

• SpectorConfigNotFoundException — Thrown when an explicitly required configuration file is missing from the file system.
• SpectorConfigParseException — Thrown when a YAML file contains malformed syntax, unescaped characters, or illegal indentation.
• SpectorValidationException (with ErrorCode.INVALID_ARGUMENT) — Thrown when a value violates semantic constraints (e.g. dimensions <= 0, unknown provider enum, or decay-factor > 1.0).

🚀 Next Steps¶

• Consult the Master spector.yml Reference for full configuration keys, ranges, and schema details.
• Review Environment Variables & Secrets for production container operations.
• Inspect Deployment & Cloud Config for Kubernetes Helm and Terraform setups.