feat: add content targets and loop expansion for target templates
Content targets write literal text to files via 'content:' field, without requiring an AI provider or API keys. They are not archived when overwritten. Loop expansion allows defining 'loops:' at the top level with named lists of values. Targets with [var] in their name are expanded via cartesian product. Variables are substituted in all string fields. Explicit targets override expanded ones. Escaping: \[var] -> [var]. Expansion happens at config load time so the rest of the system (builder, graph, state) sees only expanded targets.
This commit is contained in:
parent
bb03975ece
commit
7503672942
7 changed files with 581 additions and 2 deletions
|
|
@ -50,7 +50,8 @@ main.py # Entry point: imports and runs hokusai.cli.app
|
|||
hokusai/
|
||||
__init__.py
|
||||
cli.py # Typer CLI: build, regenerate, clean, graph, init, models commands
|
||||
config.py # Pydantic models for YAML config
|
||||
config.py # Pydantic models for YAML config + loop expansion at load time
|
||||
expand.py # Loop variable extraction, substitution, and target expansion
|
||||
graph.py # networkx DAG construction and traversal
|
||||
builder.py # Build orchestrator: incremental + parallel
|
||||
state.py # .hokusai.state.yaml hash tracking
|
||||
|
|
@ -71,7 +72,7 @@ hokusai/
|
|||
### Data flow
|
||||
|
||||
1. **cli.py** finds the `*.hokusai.yaml` in cwd, calls `load_config()` from `config.py`
|
||||
2. **config.py** parses YAML into `ProjectConfig` (pydantic), which contains `Defaults` and `dict[str, TargetConfig]`
|
||||
2. **config.py** parses YAML, expands loop templates via `expand.py` (cartesian product), then validates into `ProjectConfig` (pydantic) which contains `Defaults`, `loops`, and `dict[str, TargetConfig]`
|
||||
3. **graph.py** builds an `nx.DiGraph` from target dependencies. `get_build_order()` uses `nx.topological_generations()` to return parallel batches
|
||||
4. **builder.py** `run_build()` iterates generations. Per generation:
|
||||
- Checks each target for dirtiness via `state.py` (SHA-256 hashes of inputs, prompt, model, extra params)
|
||||
|
|
@ -85,7 +86,9 @@ hokusai/
|
|||
- **Target type inference**: `.png/.jpg/.jpeg/.webp` = image, `.md/.txt` = text. Defined in `config.py` as `IMAGE_EXTENSIONS` / `TEXT_EXTENSIONS`.
|
||||
- **Prompt resolution**: if the `prompt` string is a path to an existing file, its contents are read; otherwise it's used as-is. Supports `{filename}` placeholders. Done in `prompt.py`.
|
||||
- **Model resolution**: `resolve.py` maps target config + defaults to a `ModelInfo` with provider, model name, and capabilities.
|
||||
- **Content targets**: targets with `content:` write literal text to the file; no provider needed, no archiving on overwrite. State tracks the content string for incremental skip.
|
||||
- **Download targets**: targets with `download:` URL are fetched via httpx; state tracks the URL for incremental skip.
|
||||
- **Loop expansion**: `loops:` defines named lists of values. Targets with `[var]` in their name are expanded via cartesian product at config load time (in `expand.py`). Only variables appearing in the target name trigger expansion. Explicit targets override expanded ones. Escaping: `\[var]` → literal `[var]`. Substitution applies to all string fields (prompt, content, download, inputs, reference_images, control_images). The rest of the system sees only expanded targets.
|
||||
- **BFL client is async**: custom async client in `providers/bfl.py` polls for completion.
|
||||
- **Mistral client is natively async**: uses `complete_async()` directly.
|
||||
- **OpenAI clients are async**: use the official `openai` SDK with async methods.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue