streamd/REQUIREMENTS.md

# Streamd Requirements

Streamd (stylized as "Strea.md") is a personal knowledge management and time-tracking CLI tool that organizes time-ordered markdown files using `@Tag` annotations.

## Core Concepts

### Shard

A **Shard** is the fundamental unit of content. It represents a section of a markdown file (paragraph, heading, list item) that can contain markers and tags.

```
Shard {
    markers: [String]       // @Tag annotations at START of content
    tags: [String]          // @Tag annotations AFTER content begins
    start_line: int
    end_line: int
    children: [Shard]       // Nested shards (hierarchical)
}
```

### LocalizedShard

A **LocalizedShard** extends Shard with temporal and dimensional placement information.

```
LocalizedShard {
    markers: [String]
    tags: [String]
    start_line: int
    end_line: int
    moment: DateTime        // When this entry was created
    location: Map<String, String>  // Dimension placements
    children: [LocalizedShard]
}
```

---

## Tag Extraction Logic

### R1: Tag Recognition Pattern

Tags are recognized by the regex pattern: `@([^\s*\x60~\[\]]+)`

A tag is `@` followed by word characters, excluding:
- Whitespace
- Asterisks `*`
- Backticks `` ` ``
- Tildes `~`
- Brackets `[]`

**Examples of valid tags:**
- `@Task`, `@Done`, `@Waiting`
- `@Timesheet`, `@Break`
- `@ProjectName`, `@Client-ABC`

### R2: Marker vs Tag Distinction

The extraction MUST distinguish between **markers** and **tags** based on their position within a block:

| Type | Position | Purpose |
|------|----------|---------|
| **Marker** | Before any non-whitespace content | Semantic classification (triggers shard creation) |
| **Tag** | After non-whitespace content | Metadata annotation (does not trigger shard creation) |

**Example:**
```markdown
@Task @Streamd Working on feature    <!-- @Task and @Streamd are MARKERS -->
Some text here @CompletedFeature     <!-- @CompletedFeature is a TAG -->
```

### R3: Marker Boundary Tracking

The extraction algorithm MUST track a "marker boundary" state:

1. Start with `marker_boundary_encountered = false`
2. While processing tokens:
   - If whitespace-only: continue (boundary not crossed)
   - If `@Tag` token found AND boundary NOT crossed: add to markers
   - If `@Tag` token found AND boundary crossed: add to tags
   - If any non-whitespace content found: set boundary = crossed

### R4: Nested Token Handling

Tag extraction MUST handle nested markdown formatting:

- Emphasis: `*@Tag*` or `_@Tag_`
- Strong: `**@Tag**` or `__@Tag__`
- Strikethrough: `~~@Tag~~`
- Links: `[@Tag](url)`

Tags inside these formatting elements are still valid and should be extracted.

### R5: Applicable Block Types

Tag extraction applies to:
- Headings (`# Heading with @Tag`)
- Paragraphs (`@Tag in paragraph`)
- Quoute Blocks (`> @Tag in Quote`)
- List items (each item can have its own markers)

---

## Parsing Logic

### R6: Heading-Based Hierarchy

The parser MUST create a hierarchical shard structure based on markdown headings.

**Algorithm for determining split level:**

1. Find the minimum heading level that either:
   - Appears 2+ times in the block list, OR
   - Has markers AND is not the first heading
2. If no such level exists, do not split (return None)

**Example:**
```markdown
# Main Title
Content here

## Section A        <!-- Split point (level 2 appears twice) -->
Section A content

## Section B        <!-- Split point -->
Section B content
```

### R7: List Item Shard Creation

Each list item with markers MUST become its own shard:

```markdown
- @Task Item one    <!-- Shard 1 -->
- @Task Item two    <!-- Shard 2 -->
- Item three        <!-- NOT a separate shard (no markers) -->
```

### R8: Shard Simplification

When building shards, apply this optimization:
- If a shard has exactly 1 child AND no markers AND no tags
- Return the child directly instead of wrapping it

---

## Dimension Placement Logic

### R9: Dimension Configuration

A **Dimension** defines a classification axis:

```
Dimension {
    display_name: String    // For UI display
    comment: String?        // Documentation
    propagate: bool         // Whether children inherit this dimension
}
```

### R10: Marker Configuration

A **Marker** defines how a tag affects dimension placement:

```
Marker {
    display_name: String
    placements: [MarkerPlacement]
}

MarkerPlacement {
    if_with: Set<String>    // Conditional: only apply if ALL these markers present
    dimension: String       // Target dimension name
    value: String?          // Value to assign (defaults to marker name)
    overwrites: bool        // Can overwrite existing placement
}
```

### R11: Conditional Placement

Placements with `if_with` conditions MUST only apply when ALL specified markers are present on the same shard.

**Example Configuration:**
```
Marker "Task" {
    placements: [
        { dimension: "task", value: "open" },
        { if_with: ["Done"], dimension: "task", value: "done" },
        { if_with: ["Waiting"], dimension: "task", value: "waiting" },
    ]
}
```

**Behavior:**
- `@Task` alone → `task: "open"`
- `@Task @Done` → `task: "done"` (conditional overrides default)
- `@Task @Waiting` → `task: "waiting"`

### R12: Localization Algorithm

The localization process MUST follow this algorithm:

```
function localize_shard(shard, config, propagated_from_parent, moment):
    position = copy(propagated_from_parent)  // Start with inherited
    private_position = {}                     // Non-propagating dimensions

    for marker in shard.markers:
        if marker in config.markers:
            for placement in marker.placements:
                // Check conditional
                if placement.if_with is subset of shard.markers:
                    dimension = config.dimensions[placement.dimension]
                    value = placement.value OR marker

                    // Check if we can apply this placement
                    target = dimension.propagate ? position : private_position
                    if placement.dimension not in target OR placement.overwrites:
                        target[placement.dimension] = value

    // Recursively localize children with propagating dimensions
    children = [
        localize_shard(child, config, position, moment)
        for child in shard.children
    ]

    // Merge private dimensions into final position
    position.update(private_position)

    return LocalizedShard(
        markers: shard.markers,
        tags: shard.tags,
        location: position,
        moment: moment,
        children: children,
    )
```

### R13: Dimension Propagation

When `propagate = true`:
- Children inherit the dimension value from their parent
- Child can override with their own placement

When `propagate = false`:
- Dimension value is NOT inherited by children
- Each shard must have its own marker to be placed in this dimension

**Example:**
```
dimensions: {
    "project": { propagate: true },   // Children inherit project
    "task": { propagate: false },     // Each task is independent
}
```

```markdown
# @Project-X
## @Task Item A     <!-- project: "Project-X", task: "open" -->
### Sub-item        <!-- project: "Project-X", task: (none) -->
## @Task Item B     <!-- project: "Project-X", task: "open" -->
```

### R14: Overwrite Behavior

Default: A placement does NOT overwrite an existing value in the dimension.

With `overwrites: true`: The placement WILL replace any existing value.

This allows conditional placements to override base placements.

---

## File Naming Convention

### R15: File Name Format

Files follow the pattern: `YYYYMMDD-HHMMSS [markers].md`

- `YYYYMMDD`: Date (8 digits, required)
- `HHMMSS`: Time (4-6 digits, optional, pads with zeros)
- `[markers]`: Space-separated marker names extracted from file content

**Extraction regex:** `^(?P<date>\d{8})(?:-(?P<time>\d{4,6}))?.+\.md$`

### R16: Temporal Markers

Special markers can override the file timestamp:

- Date markers: `@YYYYMMDD` (8 digits)
- Time markers: `@HHMMSS` (6 digits)

These are used for entries that reference a different time than when the file was created.

---

## Timesheet Module

### R17: Timesheet Point Types

```
TimesheetPointType {
    Card,       // Clock in / start work
    Break,      // Clock out / end work
    SickLeave,
    Vacation,
    Holiday,
    Undertime,
}
```

### R18: Timesheet State Machine

Process timesheet shards chronologically per day:

1. Start state: "on break" (not working)
2. `Card` marker: Transition to "working", record start time
3. `Break` marker: Transition to "on break", emit timecard from previous start to now
4. Special markers (SickLeave, Vacation, etc.): Set day type

**Validation:** The last entry of each day MUST be a `Break` (cannot end day while working).

---

## Query System

### R19: Shard Search

Provide recursive search through the shard tree:

- `find_shard(predicate)`: Find all shards matching a predicate function
- `find_by_position(dimension, value)`: Find shards where `location[dimension] == value`
- `find_by_set_dimension(dimension)`: Find shards where dimension exists in location

---

## CLI Commands

### R20: Core Commands

| Command | Description |
|---------|-------------|
| `streamd new` | Create new timestamped file, open editor, rename with markers on close |
| `streamd todo` | List all shards with `task: "open"` |
| `streamd edit [n]` | Edit nth file (supports negative indexing for recent files) |
| `streamd timesheet` | Extract and export timesheet data as CSV |
| `streamd completions <shell>` | Generate shell completions (bash, zsh, fish, elvish, powershell) |

---

## Application Configuration

### R22: Config File Location

The application configuration is stored at `~/.config/streamd/config.toml`:

```toml
base_folder = "/path/to/stream/files"
```

### R23: Environment Variable Override

The `STREAMD_BASE_FOLDER` environment variable can override the config file setting.

---

## Configuration Merging

### R24: Configuration Composition

Multiple configurations can be merged:

- Dimensions are combined (later configs can add new dimensions)
- Markers are combined (later configs can add new markers)
- This allows base configuration + domain-specific extensions