# **OCT Architecture Document**
**Version:** 3.3
**Status:** Authoritative Reference
**Scope:** Defines the architecture, structure, and design philosophy of the OCT (Option C Tools) ecosystem v0.25.0, depending on `oc_diagnostics` v2.0.

**Version history:**
- 1.1 — Initial release
- 1.2 — Updated Diagnostics Package section to reflect `oc_diagnostics` v1.5; added FastAPI middleware, logging handler, full public API surface, and complete `debug_config.json` schema
- 2.0 — Major update: added Formatter (v0.7.1), Health Dashboard (v0.7.2), Core module (`exclusions.py`, `compat.py`), `oct clean` command. Updated directory structure, diagrams, module inventory, and test suite documentation. Marked implemented features; updated future items.
- 2.1 — Added global `--root-dir` CLI option for overriding project root detection (v0.7.3).
- 2.2 — Added `safe_parse()` shared helper and `syntax_warnings` lint check; captures `SyntaxWarning` from AST parsing as structured lint findings (v0.7.4).
- 2.3 — Hardened `oct health` test reporting: pre-flight pytest detection, actionable error details with `--tb=line`, test count always shown, `=` decoration stripped, stderr fallback, status label formatting. Added `pytest>=7.0` as core dependency. Added `tests/run_tests.py` and `tests/Tests_README.md` to OCT's own test suite (v0.7.5).
- 2.4 — Added `no_hardcoded_secrets` linter rule (§6b Secret Hygiene, DD-05). Added `NEVER_EXPORT_FILE_PATTERNS` to exclusions.py for file-level export filtering. Removed `.env` from exporter extension profiles. Added `oct docs --doxygen`, `--all`, `--clean`, `--dry-run` flags and `oct/docs/oct_docs.py` module. Updated directory structure, test counts (97 tests across 4 suites), and module inventory.
- 2.5 — Fixed `oct export-source` config resolution: uses `get_project_root()` instead of raw `cwd`, three-tier config fallback (project → oct bundled → built-in defaults). Added `oct test` meta-project sub-discovery for monorepo roots. Updated CLI commands table and sequence diagram (v0.9.1).
- 2.6 — Added `oct export-skeleton` command (AST-based skeleton extraction with `--no-diagnostics`, `--single-dir`, `--verbose`, `--clean`). Added `_skeleton` to `WILDCARD_EXCLUDE_DIRS`. Updated `oct clean` to also remove `_skeleton_code-*` dirs. Updated test counts (123 tests across 5 suites).
- 3.0 — Added Git integration modules: `oct/core/git.py` (git primitives), `oct/git/` package (quality_gate, policy, audit, conventional, oct_git commands). Added `oct git status`, `oct git check`, `oct git init`, `oct git commit`, `oct git hooks` commands. Updated directory structure, diagrams, CLI table, and test counts (477 tests across 10 suites). Version bump v0.9.1 → v0.12.0.
- 3.1 — Phase 4E: Added `oct git changelog` command, `oct/git/changelog.py` module, branch naming enforcement, branch-aware auto-profile selection, `oct health` git section, `oct export-source --diff`, `oct test --changed`, `.octrc.json` git schema validation. Updated test counts (542 tests, 2 skipped). Version bump v0.12.0 → v0.13.0.
- 3.2 — Phase 6B+: Added `oct/tools/secret_scanner.py` (standalone scanner, OI-514), `oct/core/waivers.py` (shared inline waiver parsing, OI-540). Added entropy-based secret detection (FS-507) with per-context thresholds (code ≥ 4.5, docstring > 5.0, comment > 5.0). Added `entropy_exclusions` in `.octrc.json` `secret_scanner` section (field-name exclusions for namedtuple/dataclass fields, path-based exclusions for test directories). Quality gate honours inline waivers universally. Updated directory structure, module table, configuration schema, test counts (1402 tests, 9 skipped).
- 3.3 — Phase F1 / v0.25.0: Workspace mode introduced (`.option_c_workspace.json` manifest). `oct git status / commit / ignore / init` fan out across subprojects when invoked from the workspace root, with per-subproject quality gates and a workspace bucket for files matched by `workspace_files[]`. Per-subproject `CHANGELOG.md` and `SECURITY.md` introduced at canonical root locations. Updated `oc_diagnostics` reference to v2.0.0; updated scope line to v0.25.0.

---

# **1. Overview**

OCT (Option C Tools) is the official development ecosystem for all Option C-compliant projects.
It provides:

- a **linter** — static compliance analysis (AST-based)
- a **formatter** — auto-fix compliance violations
- a **health dashboard** — single-command project compliance report
- a **file generator** — create new compliant files
- a **project scaffolder** — create full project templates
- a **documentation validator** — check required docs
- a **test runner** — pytest wrapper
- a **source exporter** — consolidate source for review
- a **cleanup tool** — remove generated artifacts
- integration with the standalone `oc_diagnostics` package

OCT enforces the Option C coding philosophy and provides a predictable, diagnostics-aware development workflow.

---

# **2. High-Level Architecture**

```
+-------------------------------------------------------------+
|                       OCT CLI [--root-dir]                   |
|  lint | format | health | new | scaffold | docs | test      |
|  export-source | export-skeleton | clean | diag | deps     |
|  git (status | check | init | commit | hooks)              |
+-------------------------------------------------------------+
         |                    |                    |
         v                    v                    v
+--------------------+  +-----------+  +-----------------------------+
|    OCT Modules     |  | OCT Core  |  |     oc_diagnostics v2.0     |
|  - Linter          |  | - excl.   |  |  - unified_debug            |
|  - Formatter       |  | - compat  |  |  - instrumentation          |
|  - Health          |  | - root    |  |  - ui_event_logger          |
|  - Generator       |  | - git     |  |  - fastapi_middleware       |
|  - Exporter        |  +-----------+  |  - logging_handler          |
|  - Git Integration |                 |  - self-test suite          |
+--------------------+                 +-----------------------------+
         |
         v
+-----------------------------+
|     Option C Project        |
|  - src/                     |
|  - tests/                   |
|  - docs/                    |
|  - oc_diagnostics/debug_config |
+-----------------------------+
```

OCT is a **development-time** tool.
`oc_diagnostics` is a **runtime** system.
Projects consume both.

```mermaid
flowchart TD

    subgraph CLI["OCT CLI"]
        LINT["oct lint"]
        FORMAT["oct format"]
        HEALTH["oct health"]
        NEW["oct new"]
        SCAFFOLD["oct scaffold"]
        DOCS["oct docs"]
        TEST["oct test"]
        EXPORT["oct export-source"]
        SKELETON["oct export-skeleton"]
        CLEAN["oct clean"]
        DEPS["oct deps"]
        DIAG["oct diag"]
        GIT["oct git"]
    end

    subgraph CORE["OCT Core"]
        EXCLUSIONS["exclusions.py\n(shared exclude lists)"]
        COMPAT["compat.py\n(oc_diagnostics version check)"]
        ROOT["project_root.py\n(root detection)"]
        PARSING["parsing.py\n(safe AST parsing)"]
        GITCORE["git.py\n(git primitives)"]
        WAIVERS["waivers.py\n(inline waiver parsing)"]
    end

    subgraph OCT["OCT Modules"]
        LINTER["Linter\n(oct_lint.py)"]
        FORMATTER["Formatter\n(oct_format.py)"]
        HEALTHMOD["Health Dashboard\n(oct_health.py)"]
        GENERATOR["File Generator\n(new_python_file.py)"]
        SCAFFOLDER["Project Scaffolder\n(cli.py)"]
        EXPORTER["Source Exporter\n(source_exporter.py)"]
        SKELEXP["Skeleton Exporter\n(skeleton_exporter.py)"]
        GITMOD["Git Integration\n(oct_git.py, quality_gate.py,\npolicy.py, audit.py, conventional.py)"]
    end

    subgraph DIAGPKG["oc_diagnostics v2.0\n(Standalone Package)"]
        UDEBUG["unified_debug.py\n(_dbg, _adbg, domains, levels,\nruntime API, profiling)"]
        INSTR["id_instrumentation.py\n(ID rewriting)"]
        MIDDLE["instrumentation_middleware.py\n(callback patching)"]
        LOGGER["ui_event_logger.py\n(event capture)"]
        FMIDDLE["fastapi_middleware.py\n(ASGI middleware)"]
        LHANDLER["logging_handler.py\n(stdlib logging bridge)"]
    end

    subgraph PROJECT["Option C Project"]
        SRC["src/ or ui/ or engine/"]
        TESTS["tests/"]
        DOCS2["docs/"]
        CONFIG["oc_diagnostics/debug_config.json"]
    end

    CLI --> CORE
    CLI --> OCT
    OCT --> CORE
    OCT --> DIAGPKG
    OCT --> PROJECT
    DIAGPKG --> PROJECT
```

```mermaid
mindmap
  root((OCT Ecosystem))
    CLI
      lint
      format
      health
      new
      scaffold
      docs
      test
      export-source
      export-skeleton
      clean
      deps
      diag
      git
        status
        check
        init
        commit
        hooks
    Core
      exclusions
      compat
      project_root
      git primitives
    Modules
      Linter
      Formatter
      Health Dashboard
      File Generator
      Project Scaffolder
      Source Exporter
      Git Integration
    oc_diagnostics (v2.0)
      _dbg / _adbg
      instrumentation
      middleware
      event logger
      FastAPI middleware
      logging handler
      self-test suite
    Option C Project
      src/
      tests/
      docs/
      oc_diagnostics/debug_config.json
```

---

# **3. Directory Structure of OCT**

```
oct/
    oct/
        __init__.py              # Package init, exposes __version__
        __main__.py              # python -m oct support
        cli.py                   # Click CLI — all commands
        core/
            project_root.py      # Project root detection (docs/ + tests/ + oc_diagnostics/)
            exclusions.py        # Shared exclusion lists (dirs + NEVER_EXPORT_FILE_PATTERNS)
            compat.py            # oc_diagnostics version compatibility check
            parsing.py           # Shared safe AST parsing
            terminal.py          # Shared terminal color utilities
            git.py               # Git primitives (run, staged files, branch, diff)
            waivers.py           # Shared inline waiver parsing (OCT-LINT: disable=...)
        linter/
            __init__.py
            oct_lint.py          # AST-based Option C linter (incl. no_hardcoded_secrets)
        formatter/
            __init__.py
            oct_format.py        # Auto-fixer for compliance violations
        health/
            __init__.py
            oct_health.py        # Project compliance dashboard
        deps/
            __init__.py
            oct_deps.py          # Dependency graph generation
        diag/
            __init__.py
            oct_diag.py          # Diagnostics config management
        docs/
            __init__.py
            oct_docs.py          # Documentation pipelines (Sphinx, Doxygen, clean)
        git/
            __init__.py
            oct_git.py           # oct git status/check/init/commit/hooks commands
            quality_gate.py      # Unified quality gate (lint + format + secrets)
            policy.py            # Profile resolution, protected branches, scope filtering
            audit.py             # @audited decorator, AuditRecord, JSONL logging
            conventional.py      # Conventional Commits parser, validator, diagnostics impact
        hooks/
            __init__.py
            oct_hooks.py         # Pre-commit hook integration (legacy)
        typecheck/
            __init__.py
            oct_typecheck.py     # Type checking integration
        generator/
            new_python_file.py   # Option C file generator
        tools/
            source_exporter.py   # Source code consolidation exporter
            skeleton_exporter.py # Structural skeleton exporter (AST-based)
            secret_scanner.py    # Standalone hardcoded-secret detection (AST + entropy)
        templates/
            sphinx/              # Sphinx templates (conf.py.template, CSS, logo)
    tests/
        run_tests.py             # pytest runner script
        Tests_README.md          # Test suite documentation
        tests_linter/
            conftest.py          # Shared fixtures
            fixtures/            # Test fixture files
            test_func_pattern.py
            test_header_fixing.py
            test_headers.py
            test_integration.py
            test_no_hardcoded_secrets.py  # §6b secret hygiene rule tests
            test_regression.py
        tests_formatter/
            conftest.py          # Shared fixtures
            fixtures/            # Test fixture files
            test_docstring_fixing.py
            test_func_pattern_fixing.py
            test_header_fixing.py
            test_import_fixing.py
            test_integration.py
        tests_health/
            conftest.py          # Shared fixtures
            test_docs_health.py
            test_integration.py
            test_lint_health.py
        tests_core/
            test_exclusions.py
            test_git.py              # Git primitives (run, staged, branch, diff)
            test_project_root.py
        tests_docs/
            test_docs_clean.py   # oct docs --clean tests
            test_doxygen_cleanup.py
            test_find_packages.py
        tests_git/
            conftest.py          # Shared fixtures (temp_git_project, make_compliant_py, etc.)
            test_audit.py        # Audit record, JSONL writing, @audited decorator
            test_conventional.py # CC parser, validator, error formatting, diagnostics impact
            test_git_check.py    # Quality gate command integration tests
            test_git_commit.py   # Commit command: exit codes, safety gates, secrets invariant
            test_git_hooks.py    # Hooks install/status/remove, deprecation shim
            test_git_init.py     # Init command: .gitignore, .gitattributes, workflow
            test_git_status.py   # Status command integration tests
            test_policy.py       # Profile resolution, protected branches, scope filtering
    docs/
        ARCHITECTURE.md          # This file
        ROADMAP.md
        REFERENCE.md
        README.md
        linter.md
    pyproject.toml
    README.md
    pytest.ini
```

### **3.1 Module Dependency Graph**

```mermaid
classDiagram
    class OCTCLI {
        +lint()
        +format()
        +health()
        +new()
        +scaffold()
        +docs()
        +test()
        +export_source()
        +clean()
        +git()
    }

    class Core {
        +find_project_root()
        +check_oc_diagnostics_compat()
        +LINTER_EXCLUDE_DIRS
        +WILDCARD_EXCLUDE_DIRS
    }

    class Linter {
        +run_linter()
        +validate_header_block()
        +check_docstring()
        +check_diagnostics_profile()
        +check_dbg_usage()
        +check_dbg_import()
        +check_func_pattern()
        +check_no_hardcoded_secrets()
        +find_python_files()
        +LinterContext
    }

    class Formatter {
        +run_formatter()
        +fix_header_block()
        +fix_docstring()
        +fix_dbg_import()
        +fix_func_pattern()
        +FormatterContext
    }

    class HealthDashboard {
        +run_health()
        +check_lint_health()
        +check_fixable_health()
        +check_docs_health()
        +check_test_health()
        +check_compat_health()
    }

    class Generator {
        +create_new_file()
    }

    class Exporter {
        +run_exporter()
        +clean_source_dirs()
        +clean_pycache_dirs()
    }

    class GitIntegration {
        +git_status_cmd()
        +git_check_cmd()
        +git_init_cmd()
        +git_commit_cmd()
        +hooks_install_cmd()
        +hooks_status_cmd()
        +hooks_remove_cmd()
        +run_quality_gate()
        +validate_commit_message()
    }

    OCTCLI --> Core
    OCTCLI --> Linter
    OCTCLI --> Formatter
    OCTCLI --> HealthDashboard
    OCTCLI --> Generator
    OCTCLI --> Exporter
    OCTCLI --> GitIntegration
    Linter --> Core
    Formatter --> Core
    Formatter --> Linter : reuses check functions
    HealthDashboard --> Linter : reuses check functions
    HealthDashboard --> Formatter : reuses fix functions (dry-run)
    HealthDashboard --> Core
    Exporter --> Core
    GitIntegration --> Core
    GitIntegration --> Linter : quality gate
    GitIntegration --> Formatter : quality gate
```

---

# **4. OCT CLI**

The OCT CLI is the primary entry point for all Option C tooling. It uses Click for
command routing and delegates all logic to the appropriate module.

### **Available Commands**

| Command | Module | Description |
|---------|--------|-------------|
| `oct lint` | `oct.linter.oct_lint` | Run the Option C linter (AST-based, JSON output, rule profiles) |
| `oct format` | `oct.formatter.oct_format` | Auto-fix compliance violations (headers, docstrings, imports, func patterns) |
| `oct health` | `oct.health.oct_health` | Display project compliance dashboard (lint, fixable, docs, tests, compat) |
| `oct new` | `oct.generator.new_python_file` | Create a new Option C-compliant Python file |
| `oct scaffold` | `oct.cli` (inline) | Create a full Option C project template |
| `oct docs` | `oct.docs.oct_docs` | Validate docs, generate Sphinx/Doxygen HTML, or clean generated artifacts |
| `oct test` | `oct.cli` (`_run_pytest`) | Run project tests via pytest (single-project or meta-project sub-discovery) |
| `oct export-source` | `oct.tools.source_exporter` | Consolidate source code for review (excludes secret-bearing files) |
| `oct export-skeleton` | `oct.tools.skeleton_exporter` | Export structural skeletons (headers, docstrings, signatures) for AI inspection |
| `oct clean` | `oct.tools.source_exporter`, `oct.tools.skeleton_exporter` | Remove `_source_code-*`, `_skeleton_code-*`, and `__pycache__/` directories |
| `oct diag` | `oct.diag.oct_diag` | Validate, inspect, and update `debug_config.json` |
| `oct deps` | `oct.deps.oct_deps` | Dependency graph generation (JSON, Mermaid, DOT) |
| `oct git status` | `oct.git.oct_git` | Show git status with Option C compliance annotations |
| `oct git check` | `oct.git.oct_git` | Run the unified quality gate (lint + format + secrets) |
| `oct git init` | `oct.git.oct_git` | Initialise or enhance a git repo with Option C conventions |
| `oct git commit` | `oct.git.oct_git` | Commit with quality gate and non-bypassable secrets enforcement |
| `oct git hooks` | `oct.git.oct_git` | Install, verify, or remove Option C git hooks |

Each command is implemented as a Click subcommand registered on the `cli` group.

### **CLI Design Patterns**

- **Argparse passthrough** (`lint`, `format`): Uses `@click.argument("args", nargs=-1)` to forward flags to internal argparse within the module. Allows richer flag handling.
- **Click options** (`health`, `new`, `scaffold`, `clean`): Uses `@click.option()` directly for simpler command interfaces.

---

# **5. OCT Modules**

## **5.1 Core (`oct/core/`)**

Shared utilities used by multiple OCT modules.

| File | Responsibility |
|------|----------------|
| `project_root.py` | Walk upward from cwd to find the project root (directory containing `docs/`, `tests/`, and `oc_diagnostics/` or `diagnostics/`). |
| `exclusions.py` | Shared `LINTER_EXCLUDE_DIRS`, `WILDCARD_EXCLUDE_DIRS`, and `NEVER_EXPORT_FILE_PATTERNS` sets, used by linter, formatter, health, exporter, and docs. |
| `compat.py` | Check whether `oc_diagnostics` is installed and version-compatible. Returns a warning message or empty string. |
| `waivers.py` | Shared inline waiver parsing (`# OCT-LINT: disable=<rule> reason="..."`). Extracted from `oct_lint.py` (OI-540) so both linter and quality gate can import the same parser without the quality gate depending on the linter. |

---

## **5.2 Linter (`oct/linter/oct_lint.py`)**

Responsibilities:

- Enforce Option C header rules (3-line header + blank line)
- Enforce module docstring structure (Purpose, Responsibilities, Diagnostics, Contracts)
- Enforce diagnostics profile block (Domain + L2/L3/L4)
- Enforce `_dbg` usage for non-trivial modules (AST-verified, > 20 lines)
- Enforce `from oc_diagnostics import _dbg` canonical import (AST-verified)
- Enforce `func = "function_name"` pattern per function (AST-verified)
- Detect hardcoded secrets in assignments, dicts, and function defaults (§6b Secret Hygiene, AST-verified) — delegates to `oct.tools.secret_scanner` (OI-514)
- Detect high-entropy string literals via Shannon entropy (FS-507) — per-context thresholds: code strings ≥ 4.5 bits/char, docstrings > 5.0, comments > 5.0 (reserved)
- Support `entropy_exclusions` from `.octrc.json` — field-name exclusions for namedtuple/dataclass documentation fields, path-based exclusions for test directories
- Support inline waivers (`# OCT-LINT: disable=<rule> reason="..."`) — parsed by `oct.core.waivers` (OI-540)
- Check for regression tests (heuristic)
- Generate terminal reports with colour, JSON output, and timestamped log files
- Support configurable rule profiles via `.octrc.json` (`strict`, `compact`, `proto`)

The linter is **static-analysis only** (AST parsing + text analysis). It never imports
project modules.

Key exports: `run_linter()`, `validate_header_block()`, `check_docstring()`,
`check_diagnostics_profile()`, `check_dbg_usage()`, `check_dbg_import()`,
`check_func_pattern()`, `check_no_hardcoded_secrets()` (re-exported from `oct.tools.secret_scanner`),
`find_python_files()`, `LinterContext`.

---

## **5.3 Formatter (`oct/formatter/oct_format.py`)**

*Added in v0.7.1.*

Responsibilities:

- Auto-fix malformed header blocks (shebang, encoding, file identity)
- Insert or complete module docstrings with required sections
- Add canonical `from oc_diagnostics import _dbg` import
- Insert `func = "function_name"` in functions that call `_dbg()`
- Archive originals to `.formatter_archive/<filename>.<timestamp>.bak` before modification
- Support dry-run mode (default), fix mode, JSON output, and verbose output

Design:

- **Report-then-fix model**: Default is dry-run (report only). `--fix` applies changes.
- **Fixer pipeline**: header -> docstring -> import -> func_pattern (applied sequentially per file).
- Each fixer returns `Tuple[bool, List[str]]` (changed, messages).
- Reuses linter's `find_python_files()` and exclusion lists.
- Uses `FormatterContext` dataclass to encapsulate tool state.

---

## **5.4 Health Dashboard (`oct/health/oct_health.py`)**

*Added in v0.7.2.*

Responsibilities:

- Aggregate lint check results across all Python files (per-check pass counts, compliance %)
- Count fixable violations by running formatter checks in dry-run mode
- Validate presence of required documentation files
- Run pytest via subprocess and capture pass/fail status with test count and actionable error details
- Detect missing pytest before subprocess invocation
- Check `oc_diagnostics` version compatibility
- Output results as terminal report or JSON

Design:

- **Composition layer**: Reuses existing linter check functions and formatter fix functions
  directly (no subprocess calls), giving structured data natively.
- Five independent check functions: `check_lint_health()`, `check_fixable_health()`,
  `check_docs_health()`, `check_test_health()`, `check_compat_health()`.
- All checks are independent; failure of one does not prevent others.
- Terminal output uses ASCII-safe characters for Windows compatibility.

---

## **5.5 File Generator (`oct/generator/new_python_file.py`)**

Responsibilities:

- Create new Python files with correct Option C headers
- Insert `from oc_diagnostics import _dbg` into generated files
- Generate module docstring templates with all required sections
- Ensure correct file identity paths relative to project root
- Validate target path is inside project root
- Support `--force` (overwrite) and `--dry-run` modes

---

## **5.6 Source Exporter (`oct/tools/source_exporter.py`)**

Responsibilities:

- Consolidate all source files into `_source_code-<timestamp>` directories
- Generate `_full_source.<root>.txt` rollup
- Exclude secret-bearing files via `NEVER_EXPORT_FILE_PATTERNS` (§6b Secret Hygiene)
- Support `--verbose`, `--clean`, and `--single-dir` modes
- Handle I/O errors (including Windows MAX_PATH) gracefully: warn and continue
- Truncate output filenames exceeding 240 characters, appending an MD5 suffix for uniqueness
- Report skipped directories in the summary rather than crashing
- Support per-project config via `_oct_exporter_config.json` with fallback to default configuration.

---

## **5.7 Documentation Pipelines (`oct/docs/oct_docs.py`)**

Responsibilities:

- **Sphinx pipeline** (`--sphinx`): Set up Sphinx config, discover markdown, generate API reference via `sphinx-apidoc`, generate dependency graphs, write `index.rst`, run `sphinx-build`
- **Doxygen pipeline** (`--doxygen`): Locate existing Doxyfile, patch stale `INPUT` path in temp copy, run `doxygen`
- **Combined pipeline** (`--all`): Run both Sphinx and Doxygen sequentially
- **Clean pipeline** (`--clean`): Remove generated artifacts (`html/`, `doctrees/`, `_api/`, `doxygen_output/`, `index.rst`, `dependencies.rst`) while preserving user-authored files
- Exclude `.env*` from Sphinx via `conf.py.template` exclude_patterns

Design:

- Doxygen runs on a temporary copy of the Doxyfile — the original is never modified.
- Sphinx config (`docs/_sphinx/conf.py`) is generated once from templates; not overwritten on subsequent builds.
- Clean preserves `_sphinx/`, `_static/`, `_templates/`, `doxygen_work_dir/`, and user-authored `.md` files.

## **5.8 Skeleton Exporter (`oct/tools/skeleton_exporter.py`)**

Responsibilities:

- Extract structural skeletons from Python files via AST: headers, module docstrings, class/function signatures with decorators and type hints.
- Produce one-line placeholders for non-Python files (filename, line count, size).
- Strip named docstring sections (`--no-diagnostics` removes the Diagnostics block).
- Reuse source exporter infrastructure (config loading, exclusions, file collection, path safety).

Design:

- Uses `safe_parse()` from `oct.core.parsing` for robust AST parsing.
- Walks `tree.body` (not flat `ast.walk`) to preserve structural nesting (classes contain methods).
- Uses `ast.unparse()` for faithful signature reconstruction (annotations, defaults, decorators).
- `_strip_docstring_section()` is generalised by section name, enabling future `--no-contracts` or `--sections-exclude` flags.
- Output structure mirrors `source_exporter`: per-directory `_skeleton_code-*` dirs with `_skeleton.<dotpath>.txt` files plus `_full_skeleton.<root>.txt` rollup.

## **5.9 Git Integration (`oct/git/`)**

*Added in v0.11.0 (Phase 4A/4B), extended in v0.11.1 (4C), v0.12.0 (4D), v0.13.0 (4E).*

Responsibilities:

- Git primitives: run commands, detect repo, list staged files, current branch, diffs
- Quality gate: unified lint + format + secrets check with profile-based policy
- Audit trail: `@audited` decorator for JSONL logging of all git operations
- Conventional Commits: parser, validator, diagnostics impact analysis
- Commit workflow: pre-flight checks, secrets enforcement (non-bypassable), protected-branch guards
- Hooks management: install/status/remove `.pre-commit-config.yaml` hooks

Architecture:

| File | Responsibility |
|------|----------------|
| `oct/core/git.py` | Low-level git primitives (subprocess wrappers, no OCT policy). `git_run()`, `git_staged_files()`, `current_branch()`, `is_git_repo()`. |
| `oct/git/quality_gate.py` | Orchestrates lint, format, secrets checks. Returns `QualityGateResult` with violation counts. Supports `entropy_exclusions` (field-name + path-based), inline waivers (OI-540). |
| `oct/git/policy.py` | Profile resolution (`resolve_effective_profile`), protected-branch detection, scope filtering, `apply_profile_policy`. |
| `oct/git/audit.py` | `AuditRecord` dataclass, `@audited` decorator, JSONL file writer. All `oct git` commands are audited. |
| `oct/git/conventional.py` | `ConventionalCommit` dataclass, `parse_commit_message`, `validate_commit_message`, `analyse_diagnostics_impact`. |
| `oct/git/oct_git.py` | Click commands: `status`, `check`, `init`, `commit`, `hooks` subgroup (`install`/`status`/`remove`). |

Key design decisions:

- **Zero external dependencies**: Git commands use `subprocess.run` (not gitpython).
- **Non-bypassable secrets (G-D4 invariant)**: `--force` and `--no-verify` never skip secrets detection. The quality gate always runs, and `result.secrets_findings` is checked unconditionally.
- **Protected branches**: `main`/`master` (configurable via `.octrc.json`) force strict profile regardless of CLI override. Direct commits are blocked in strict mode.
- **Audit trail**: Every `oct git` command is decorated with `@audited(cmd_name)`, which writes a JSONL record to `logs/oct_git_audit*.jsonl` with timestamp, command, exit code, duration, and command-specific metadata.
- **Commit exit codes differ from quality gate exit codes**: Quality gate uses 0–5 (per-check granularity). Commit uses 0–4 (outcome-oriented). The commit command maps quality gate results to its own scheme.

---

# **6. Configuration**

### **6.1 Per-project linter configuration (`.octrc.json`)**

```json
{
  "linter": {
    "profile": "default",
    "exclude_dirs_add": ["vendor"],
    "exclude_dirs_remove": ["data"],
    "wildcard_exclude_add": ["_temp"],
    "rules": {
      "func_pattern": false
    }
  }
}
```

Profiles: `strict` (all rules), `compact` (public functions), `proto` (headers only).
Individual rule overrides via `rules` map.

Available rules: `header`, `docstring`, `diagnostics_profile`, `dbg_usage`, `dbg_import`,
`func_pattern`, `no_hardcoded_secrets`, `tests_exist`, `type_hints`.

### **6.2 Secret scanner configuration (`secret_scanner` in `.octrc.json`)**

```json
{
  "secret_scanner": {
    "code_entropy_threshold": 4.5,
    "docstring_entropy_threshold": 5.0,
    "comment_entropy_threshold": 5.0,
    "entropy_exclusions": [
      {
        "field_name_pattern": "correct_pattern",
        "context": "namedtuple_field",
        "reason": "Documentation code examples in rule definitions"
      },
      {
        "path_pattern": "tests/",
        "context": "test_directory",
        "reason": "Test fixtures exempt from entropy",
        "restrictions": {
          "require_waiver_for_secret_patterns": true,
          "exempt_entropy_only": true
        }
      }
    ]
  }
}
```

- `code_entropy_threshold` — Shannon entropy threshold for code strings (≥ comparison). Set to 0 to disable entropy checking. Default 4.5.
- `docstring_entropy_threshold` — threshold for docstrings (strict > comparison). Default 5.0.
- `comment_entropy_threshold` — reserved for future AST-level comment scanning. Default 5.0.
- `entropy_exclusions` — array of exclusion rules:
  - `context: "namedtuple_field"` + `field_name_pattern` — exempt keyword arguments with matching names in Call nodes (e.g. `RuleInfo(correct_pattern="...")`).
  - `context: "test_directory"` + `path_pattern` — exempt files under matching paths. `restrictions.exempt_entropy_only` disables entropy without disabling name-pattern detection. `restrictions.require_waiver_for_secret_patterns` requires inline waivers for name-based findings.

**Known limitation:** Inline waivers (`# OCT-LINT: disable=no_hardcoded_secrets reason="..."`) are matched at file level — a single waiver suppresses all `no_hardcoded_secrets` findings in the file. Per-line waiver matching is a planned improvement.

### **6.3 Per-project exporter configuration (`_oct_exporter_config.json`)**

Supports add/remove/replace semantics for `include_extensions`, `exclude_dirs`, and
`wildcard_exclude_dirs`. When missing, oct export-source uses the default configuration
defined in `oct/core/exporter_config.py`.

---

# **7. oc_diagnostics Integration**

`oc_diagnostics` is a standalone runtime package providing:

- `_dbg` — synchronous structured debug logging
- `_adbg` — async variant for use inside `async def` functions (FS-16)
- `DEBUG_LEVELS` — dict of domain to active level
- `init()` — activate stream interception and exception hook; accepts `watch_config=True` for hot-reload (FS-09)
- `set_debug_level(domain, level)` — runtime domain level override (FS-03)
- `set_global_setting(key, value)` — runtime global setting override (FS-03)
- `dbg_timed(domain, func, label, level)` — context manager: logs ENTER + EXIT with elapsed ms (FS-12)
- `dbg_scope(domain, func, label, level)` — context manager: logs ENTER + EXIT without timing (FS-12/FS-19)
- `dbg_profile(domain, func, label, level)` — decorator: wraps a function with `dbg_timed` (FS-12)
- `OcDiagnosticsHandler` — Python `logging.Handler` that routes records through `_dbg` (FS-08)
- `apply_instrumentation` — Dash ID instrumentation (requires `pip install oc_diagnostics[dash]`)
- `DiagnosticsMiddleware` — pure ASGI HTTP middleware (requires `pip install oc_diagnostics[fastapi]`)

```mermaid
flowchart TD

    CONFIG["oc_diagnostics/debug_config.json\n(Project-specific)"]

    UDEBUG["unified_debug.py\n(_dbg, _adbg, domains, levels,\nruntime API, profiling, hot-reload)"]
    LHANDLER["logging_handler.py\n(OcDiagnosticsHandler)"]
    INSTR["id_instrumentation.py\n(ID rewriting)"]
    MIDDLE["instrumentation_middleware.py\n(callback patching)"]
    LOGGER["ui_event_logger.py\n(event capture)"]
    FMIDDLE["fastapi_middleware.py\n(ASGI middleware)"]

    APP["Dash / FastAPI / Application Code"]
    CALLBACKS["Callbacks"]
    OUTPUT["Instrumented Output"]

    CONFIG --> UDEBUG
    UDEBUG --> INSTR
    UDEBUG --> MIDDLE
    UDEBUG --> LOGGER
    UDEBUG --> LHANDLER
    UDEBUG --> FMIDDLE

    APP --> CALLBACKS
    CALLBACKS --> MIDDLE
    MIDDLE --> OUTPUT
    INSTR --> OUTPUT
    LOGGER --> OUTPUT
```

### **Project structure**
Projects include only:

```
oc_diagnostics/debug_config.json
```

Everything else is imported from `oc_diagnostics`.

### **Installation**

Core (always required):

```bash
pip install oc_diagnostics
```

With Dash instrumentation:

```bash
pip install oc_diagnostics[dash]
```

With FastAPI / ASGI middleware:

```bash
pip install oc_diagnostics[fastapi]
```

With JSON Schema config validation (FS-18):

```bash
pip install oc_diagnostics[validation]
```

### **Import surface**

Top-level imports (always available):

```python
from oc_diagnostics import _dbg                    # synchronous structured logging
from oc_diagnostics import _adbg                   # async variant (FS-16)
from oc_diagnostics import DEBUG_LEVELS
from oc_diagnostics import init
from oc_diagnostics import set_debug_level         # runtime domain level override (FS-03)
from oc_diagnostics import set_global_setting      # runtime global setting override (FS-03)
from oc_diagnostics import dbg_timed               # timing context manager (FS-12)
from oc_diagnostics import dbg_scope               # lifecycle context manager (FS-12/FS-19)
from oc_diagnostics import dbg_profile             # timing decorator (FS-12)
from oc_diagnostics import OcDiagnosticsHandler    # Python logging integration (FS-08)
from oc_diagnostics import apply_instrumentation   # requires: pip install oc_diagnostics[dash]
from oc_diagnostics import DiagnosticsMiddleware   # requires: pip install oc_diagnostics[fastapi]
```

Sub-module imports for advanced usage:

```python
from oc_diagnostics.id_instrumentation import instrument_ids, MAX_INSTRUMENT_DEPTH
from oc_diagnostics.ui_event_logger import register_callbacks, REQUIRED_STORE_ID
from oc_diagnostics.unified_debug import ADDITIONAL_STRUCTURAL_PREFIXES, ADDITIONAL_STRUCTURAL_EXACT
from oc_diagnostics.unified_debug import HTTP_REDACT_HEADERS, UI_EVENT_PROPERTIES
```

### **debug_config.json — full schema**

All keys available in `global_settings`:

```json
{
  "domains": {
    "GENERAL": { "level": 2, "color": "default" },
    "UI":      { "level": 3, "color": "cyan" },
    "DATA":    { "level": 3, "color": "green" },
    "HTTP":    { "level": 2, "color": "blue" },
    "STDOUT":  { "level": 2, "color": "gray" },
    "STDERR":  { "level": 2, "color": "red" },
    "PYERR":   { "level": 1, "color": "red" }
  },
  "global_settings": {
    "silent_mode": false,
    "enable_timestamps": true,
    "enable_colors": true,
    "include_filename": true,
    "output_mode": "both",
    "instrumentation_enabled": false,
    "max_log_files": 10,
    "max_log_size_mb": 5.0,
    "ui_event_properties": ["n_clicks", "value", "data", "figure", "selectedData"],
    "instrumentation_allow_all": false,
    "mode": "dev",
    "http_redact_headers": ["authorization", "cookie", "set-cookie", "x-api-key"],
    "structural_id_prefixes": [],
    "structural_id_exact": []
  }
}
```

`output_mode` options: `"terminal"`, `"file"`, `"both"`, `"json"` (JSONL structured log, FS-07).

`mode` options: `"dev"` (default), `"prod"` / `"production"` / `"safe"` (suppresses stream interception and exception hook, FS-15). Can also be set via `OC_DIAGNOSTICS_MODE` environment variable.

Environment variables that override config:
- `OC_DIAGNOSTICS_CONFIG` — absolute path to `debug_config.json`
- `OC_DIAGNOSTICS_LOG_DIR` — absolute path for log directory
- `OC_DIAGNOSTICS_MODE` — set to `prod` to activate production mode
- `UNIFIED_DEBUG_DISABLE_EXCEPTION_HOOK=1` — disable custom exception hook
- `UNIFIED_DEBUG_DISABLE_STDIO_INTERCEPT=1` — disable stdout/stderr interception

### **OCT Integration**
- `oct scaffold` creates `oc_diagnostics/debug_config.json` with a valid `oc_diagnostics` template
- `oct new` generates files with `from oc_diagnostics import _dbg`
- `oct lint` checks `_dbg` import source, usage, and diagnostics profile blocks
- `oct format` adds missing `_dbg` imports and `func = "..."` patterns
- `oct health` checks `oc_diagnostics` version compatibility
- `oct docs` validates diagnostics blocks
- Future: `oct diag` will expose diagnostics tools directly

---

# **8. Option C Coding Rules (Summary)**

### **8.1 File headers**
- shebang (`#!/usr/bin/env python3`)
- encoding (`# -*- coding: utf-8 -*-`)
- file identity (`# project/path.py`)
- blank line

### **8.2 Module docstrings**
- Purpose
- Responsibilities
- Diagnostics (Domain + L2/L3/L4)
- Contracts

### **8.3 Diagnostics**
- `from oc_diagnostics import _dbg` (canonical import)
- `func = "function_name"` in every function that calls `_dbg()`
- `_dbg(domain, func, message, level)` calls at L2/L3/L4
- Domain must exist in `debug_config.json`

### **8.4 Trivial file exemption**
Files under 20 lines are exempt from `_dbg` usage, `_dbg` import, and `func = "..."`
checks. This prevents over-enforcement on scaffolding files like `__init__.py`.

### **8.5 Testing**
- Regression tests required for every feature
- All tests in `<project root>/tests/`
- Tests must be deterministic
- Documented in `Tests_README.md`
- `run_tests.py` script must exist

### **8.6 Architecture**
- `docs/ARCHITECTURE.md` required
- Versioned
- Updated with significant changes

---

# **9. Test Suite**

OCT has 1402 tests across its test suites (9 skipped):

| Suite | Directory | Tests | Coverage |
|-------|-----------|-------|----------|
| Core | `tests/tests_core/` | 58+ | Project root detection, exclusions, git primitives, octrc loading, path utilities |
| Linter | `tests/tests_linter/` | 78+ | Header validation, func pattern (AST), header fixing, no_hardcoded_secrets (§6b), type hints, config validation, lint --changed, integration, regression |
| Formatter | `tests/tests_formatter/` | 42+ | Header fixing, docstring fixing, import fixing, func pattern fixing, archive rotation, format --changed, integration |
| Health | `tests/tests_health/` | 22+ | Lint aggregation, docs validation, JSON/terminal output, compat check, integration |
| Docs | `tests/tests_docs/` | 24+ | docs --clean, doxygen cleanup, package discovery, Sphinx write safety |
| Tools | `tests/tests_tools/` | 81 | Skeleton extraction (AST, headers, signatures, classes, decorators), source exporter, secret scanner (Shannon entropy, per-context thresholds, field-name exclusions, validation) |
| Git — Audit | `tests/tests_git/` | 25+ | Audit record creation, JSONL writing, @audited decorator |
| Git — Conventional | `tests/tests_git/` | 35+ | Conventional Commits parser, validator, error formatting, diagnostics impact |
| Git — Commands | `tests/tests_git/` | 118+ | Status, check, init, commit, hooks — exit codes, safety gates, integration |
| Git — Policy | `tests/tests_git/` | 54+ | Profile resolution, protected branches, scope filtering |

(9 tests skipped: platform-specific or optional-dependency tests.)

All test suites use pytest with shared `conftest.py` fixtures providing temporary
Option C project directories with the required structure.

---

# **10. OCT Project Lifecycle**

```
oct scaffold myproject
cd myproject
oct git init                     # .gitignore, .gitattributes, CI workflow
oct git hooks install            # Install pre-commit hooks
oct new src/module.py
oct docs
oct lint
oct format --fix
oct health
oct test
oct git commit -m "feat: initial implementation"
oct export-source
oct export-skeleton --no-diagnostics
```

OCT enforces correctness at every step.

```mermaid
sequenceDiagram
    participant Dev as Developer
    participant OCT as OCT CLI
    participant Project as Option C Project
    participant Diag as oc_diagnostics

    Dev->>OCT: oct scaffold myproject
    OCT->>Project: Create project structure
    OCT->>Project: Add oc_diagnostics/debug_config.json

    Dev->>OCT: oct git init
    OCT->>Project: Create .gitignore, .gitattributes, CI workflow

    Dev->>OCT: oct git hooks install
    OCT->>Project: Generate .pre-commit-config.yaml

    Dev->>OCT: oct new src/module.py
    OCT->>Project: Generate Option C file with headers, import & docstring

    Dev->>OCT: oct lint
    OCT->>Project: Run Option C linter (AST-based)

    Dev->>OCT: oct format --fix
    OCT->>Project: Auto-fix compliance violations

    Dev->>OCT: oct health
    OCT->>Project: Aggregate lint, fixable, docs, tests, compat

    Dev->>OCT: oct test
    OCT->>Project: Run test suite (or discover and run sub-project suites)

    Dev->>OCT: oct git commit -m "feat: initial"
    OCT->>Project: Quality gate + secrets check + commit

    Dev->>OCT: oct export-source
    OCT->>Project: Generate source export

    Dev->>OCT: oct export-skeleton
    OCT->>Project: Generate structural skeletons (headers, signatures, docstrings)

    Project->>Diag: Import _dbg, instrumentation, middleware
```

---

# **11. Future Enhancements**

Implemented:

- ✅ `oct new --test` — auto-generate companion test file alongside module (FS-203) — v0.8.0
- ✅ `oct diag` — diagnostics tools: validate-config, list-domains, set-level (FS-204) — v0.8.0
- ✅ `oct lint --changed` — lint only git-modified files (FS-209) — v0.8.0
- ✅ `oct docs --sphinx` — full Sphinx documentation pipeline — v0.8.1
- ✅ `oct docs --doxygen` — Doxygen integration with INPUT path patching
- ✅ `oct docs --all` — combined Sphinx + Doxygen generation
- ✅ `oct docs --clean` — remove generated documentation artifacts
- ✅ `no_hardcoded_secrets` — AST-based secret detection linter rule (§6b)
- ✅ `NEVER_EXPORT_FILE_PATTERNS` — file-level export exclusion (Layer 2, DD-05)
- ✅ `oct export-skeleton` — AST-based skeleton export (headers, docstrings, signatures) with `--no-diagnostics` — v0.9.1
- ✅ `oct git status` — Git status with OCT compliance annotations — v0.11.0
- ✅ `oct git check` — Unified quality gate (lint + format + secrets) — v0.11.0
- ✅ `oct git init` — Repository initialisation with Option C conventions — v0.11.1
- ✅ `oct format --changed` — Format only git-modified files — v0.11.1
- ✅ `oct git commit` — Conventional Commits with non-bypassable secrets — v0.12.0
- ✅ `oct git hooks` — Hook management (install/status/remove) — v0.12.0
- ✅ Pre-commit hook integration (FS-206) — v0.12.0
- ✅ `oct git changelog` — Keep a Changelog from Conventional Commits — v0.13.0
- ✅ Branch naming enforcement — advisory/blocking by profile — v0.13.0
- ✅ Branch-aware auto-profile (`pre_commit_profile: "auto"`) — v0.13.0
- ✅ `oct health` git section — 7 git health checks — v0.13.0
- ✅ `oct export-source --diff REF` — export changed files only — v0.13.0
- ✅ `oct test --changed` — run tests for changed files — v0.13.0
- ✅ `.octrc.json` git schema validation — v0.13.0
- ✅ `oct lint --explain <rule>` — inline rule documentation with registry (FS-509) — v0.20.0
- ✅ Shannon entropy-based secret detection (FS-507) — v0.21.0
- ✅ Incremental lint cache + lint baseline (FS-510, FS-508) — v0.22.0
- ✅ Standalone secret scanner extraction (OI-514) — `oct/tools/secret_scanner.py`
- ✅ Per-context entropy thresholds — code ≥ 4.5, docstring > 5.0, comment > 5.0 (reserved)
- ✅ `entropy_exclusions` in `.octrc.json` — field-name + path-based exclusions (OI-540)
- ✅ Shared inline waiver module — `oct/core/waivers.py` (OI-540)
- ✅ Quality gate honours inline waivers universally (OI-540)

Planned:

- Per-line waiver matching — waivers matched to specific finding line numbers instead of file-level suppression
- Defence-in-depth censorship module for exports/docs (`--unsafe` opt-out) — DD-05 Layer 3
- Linter plugin system for custom rules (FS-205)
- `oct deps` enhancements (FS-208)

Deferred:

- Export diff mode (FS-207)
- Debug session recorder

---

# **12. Appendix: Example OCT-Generated File**

```python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# myproject/src/example.py

"""
Example Module

Purpose
-------
Demonstrate Option C file structure.

Responsibilities
----------------
- Provide example function
- Demonstrate diagnostics usage

Diagnostics
-----------
Domain: EXAMPLE
Levels:
    L2 -- lifecycle
    L3 -- semantic details
    L4 -- deep tracing

Contracts
---------
- Must not mutate inputs
"""

from oc_diagnostics import _dbg


def example(x):
    func = "example"
    _dbg("EXAMPLE", func, f"START x={x}", 2)
    return x * 2
```