feat: add content targets and loop expansion for target templates
All checks were successful
Continuous Integration / Build Package (push) Successful in 25s
Continuous Integration / Lint, Check & Test (push) Successful in 44s

Content targets write literal text to files via 'content:' field,
without requiring an AI provider or API keys. They are not archived
when overwritten.

Loop expansion allows defining 'loops:' at the top level with named
lists of values. Targets with [var] in their name are expanded via
cartesian product. Variables are substituted in all string fields.
Explicit targets override expanded ones. Escaping: \[var] -> [var].
Expansion happens at config load time so the rest of the system
(builder, graph, state) sees only expanded targets.
This commit is contained in:
Konstantin Fickel 2026-02-21 18:39:13 +01:00
parent bb03975ece
commit 7503672942
Signed by: kfickel
GPG key ID: A793722F9933C1A5
7 changed files with 581 additions and 2 deletions

View file

@ -50,7 +50,8 @@ main.py # Entry point: imports and runs hokusai.cli.app
hokusai/
__init__.py
cli.py # Typer CLI: build, regenerate, clean, graph, init, models commands
config.py # Pydantic models for YAML config
config.py # Pydantic models for YAML config + loop expansion at load time
expand.py # Loop variable extraction, substitution, and target expansion
graph.py # networkx DAG construction and traversal
builder.py # Build orchestrator: incremental + parallel
state.py # .hokusai.state.yaml hash tracking
@ -71,7 +72,7 @@ hokusai/
### Data flow
1. **cli.py** finds the `*.hokusai.yaml` in cwd, calls `load_config()` from `config.py`
2. **config.py** parses YAML into `ProjectConfig` (pydantic), which contains `Defaults` and `dict[str, TargetConfig]`
2. **config.py** parses YAML, expands loop templates via `expand.py` (cartesian product), then validates into `ProjectConfig` (pydantic) which contains `Defaults`, `loops`, and `dict[str, TargetConfig]`
3. **graph.py** builds an `nx.DiGraph` from target dependencies. `get_build_order()` uses `nx.topological_generations()` to return parallel batches
4. **builder.py** `run_build()` iterates generations. Per generation:
- Checks each target for dirtiness via `state.py` (SHA-256 hashes of inputs, prompt, model, extra params)
@ -85,7 +86,9 @@ hokusai/
- **Target type inference**: `.png/.jpg/.jpeg/.webp` = image, `.md/.txt` = text. Defined in `config.py` as `IMAGE_EXTENSIONS` / `TEXT_EXTENSIONS`.
- **Prompt resolution**: if the `prompt` string is a path to an existing file, its contents are read; otherwise it's used as-is. Supports `{filename}` placeholders. Done in `prompt.py`.
- **Model resolution**: `resolve.py` maps target config + defaults to a `ModelInfo` with provider, model name, and capabilities.
- **Content targets**: targets with `content:` write literal text to the file; no provider needed, no archiving on overwrite. State tracks the content string for incremental skip.
- **Download targets**: targets with `download:` URL are fetched via httpx; state tracks the URL for incremental skip.
- **Loop expansion**: `loops:` defines named lists of values. Targets with `[var]` in their name are expanded via cartesian product at config load time (in `expand.py`). Only variables appearing in the target name trigger expansion. Explicit targets override expanded ones. Escaping: `\[var]` → literal `[var]`. Substitution applies to all string fields (prompt, content, download, inputs, reference_images, control_images). The rest of the system sees only expanded targets.
- **BFL client is async**: custom async client in `providers/bfl.py` polls for completion.
- **Mistral client is natively async**: uses `complete_async()` directly.
- **OpenAI clients are async**: use the official `openai` SDK with async methods.