# bulkgen

A build tool for AI-generated artifacts. Define image and text targets in a YAML config, and bulkgen handles dependency resolution, incremental builds, and parallel execution.

Uses [Mistral](https://mistral.ai) and [OpenAI](https://openai.com) for text generation, and [BlackForestLabs](https://blackforestlabs.ai) (FLUX) and [OpenAI](https://openai.com) for image generation.

## Installation

Requires Python 3.13+.

```bash
pip install .
```

Or with [uv](https://docs.astral.sh/uv/):

```bash
uv sync
```

## Quick start

1. Set your API keys:

```bash
export MISTRAL_API_KEY="your-key"
export BFL_API_KEY="your-key"
export OPENAI_API_KEY="your-key"
```

2. Create a config file (e.g. `my-project.bulkgen.yaml`):

```yaml
defaults:
  text_model: mistral-large-latest
  image_model: flux-pro

targets:
  hero.png:
    prompt: "A dramatic sunset over mountains, photorealistic"
    width: 1024
    height: 768

  blog-post.md:
    prompt: prompts/write-blog.txt
    inputs:
      - hero.png
      - notes.md
```

3. Build:

```bash
bulkgen build
```

## Config format

The config file must be named `<anything>.bulkgen.yaml` and placed in your project directory. One config file per directory.

### Top-level fields

| Field | Description |
|---|---|
| `defaults` | Default model names (optional) |
| `archive_folder` | Directory to move previous outputs into before rebuilding (optional) |
| `targets` | Map of output filenames to their configuration |

### Defaults

```yaml
defaults:
  text_model: mistral-large-latest   # used for .md, .txt targets
  image_model: flux-pro              # used for .png, .jpg, .jpeg, .webp targets
```

### Target fields

| Field | Type | Description |
|---|---|---|
| `prompt` | string | Inline prompt text, or path to a prompt file |
| `model` | string | Override the default model for this target |
| `inputs` | list[string] | Files this target depends on (other targets or existing files) |
| `reference_image` | string | Image file for image-to-image generation |
| `control_images` | list[string] | Control images (for canny/depth models) |
| `width` | int | Image width in pixels |
| `height` | int | Image height in pixels |

Target type is inferred from the file extension:
- **Image**: `.png`, `.jpg`, `.jpeg`, `.webp`
- **Text**: `.md`, `.txt`

### Prompts

Prompts can be inline strings or file references:

```yaml
targets:
  # Inline prompt
  image.png:
    prompt: "A cat sitting on a windowsill"

  # File reference (reads the file contents as the prompt)
  article.md:
    prompt: prompts/article-prompt.txt
```

If the prompt value is a path to an existing file, its contents are read. Otherwise the string is used directly.

### Dependencies

Targets can depend on other targets or on existing files in the project directory:

```yaml
targets:
  base.png:
    prompt: "A landscape scene"

  variant.png:
    prompt: "Same scene but in winter"
    reference_image: base.png    # image-to-image, depends on base.png

  summary.md:
    prompt: "Summarize these notes"
    inputs:
      - base.png                 # depends on a generated target
      - research-notes.md        # depends on an existing file
```

bulkgen resolves dependencies automatically. If you build a single target, its transitive dependencies are included.

### Archiving previous outputs

Set `archive_folder` at the top level to preserve previous versions of generated files. When a target is rebuilt, the existing output is moved to the archive folder with an incrementing numeric suffix:

```yaml
archive_folder: archive

targets:
  hero.png:
    prompt: "A dramatic sunset over mountains"
```

On each rebuild of `hero.png`, the previous file is archived as `archive/hero.01.png`, `archive/hero.02.png`, etc. The archive directory is created automatically if it doesn't exist.

## CLI

### `bulkgen build [target]`

Build all targets, or a specific target and its dependencies.

- Skips targets that are already up to date (incremental builds)
- Runs independent targets in parallel
- Continues building if a target fails (dependents of the failed target are skipped)

### `bulkgen clean`

Remove all generated target files and the build state file (`.bulkgen.state.yaml`). Input files are preserved.

### `bulkgen graph`

Print the dependency graph showing build stages:

```
Stage 0 (inputs): research-notes.md
Stage 0 (targets): base.png
Stage 1 (targets): variant.png, summary.md
  variant.png <- base.png
  summary.md <- base.png, research-notes.md
```

## Incremental builds

bulkgen tracks the state of each build in `.bulkgen.state.yaml` (auto-generated, add to `.gitignore`). A target is rebuilt when any of these change:

- Input file contents (SHA-256 hash)
- Prompt text
- Model name
- Extra parameters (width, height, etc.)

## Installation with Nix / home-manager

bulkgen provides a Nix flake with a home-manager module. Add the flake as an input and enable the module:

```nix
# flake.nix
{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    home-manager.url = "github:nix-community/home-manager";
    bulkgen.url = "github:kfickel/bulkgen";  # adjust to your actual repo URL
  };

  outputs = { nixpkgs, home-manager, bulkgen, ... }: {
    # ... your existing config, then in homeConfigurations:
    homeConfigurations."user" = home-manager.lib.homeManagerConfiguration {
      # ...
      modules = [
        bulkgen.homeManagerModules.bulkgen
        {
          programs.bulkgen.enable = true;
        }
      ];
    };
  };
}
```

This places the `bulkgen` binary on your `$PATH`. To use a different package build (e.g. from a different system or overlay), set `programs.bulkgen.package`.

The flake also exposes:

- `packages.<system>.bulkgen` — the standalone package, usable without home-manager (e.g. `nix run github:kfickel/bulkgen`)
- `devShells.<system>.default` — development shell with all dependencies

## Environment variables

| Variable | Required for |
|---|---|
| `MISTRAL_API_KEY` | Text targets via Mistral models |
| `BFL_API_KEY` | Image targets via FLUX models |
| `OPENAI_API_KEY` | Text and image targets via OpenAI models |