Timmy-time-dashboard/TESTING.md

TESTING.md

How to run tests, what each suite covers, and how to add new tests.

Quick Start

# Run the fast unit tests (recommended for development)
tox -e unit

# Run all tests except slow/external
tox -e fast

# Auto-format code before committing
tox -e format

# Lint check (CI gate)
tox -e lint

# Full CI mirror (lint + coverage)
tox -e pre-push

Prerequisites

• Python 3.11+
• tox installed (pip install tox)
• Ollama running locally (only for tox -e ollama tests)

All test dependencies are installed automatically by tox. No manual pip install needed.

Tox Environments

Command	Purpose	Speed	What It Runs
tox -e unit	Fast unit tests	~17s	@pytest.mark.unit tests, parallel, excludes ollama/docker/selenium/external
tox -e integration	Integration tests	Medium	@pytest.mark.integration tests, may use SQLite
tox -e functional	Functional tests	Slow	Real HTTP requests, no mocking
tox -e e2e	End-to-end tests	Slowest	Full system tests
tox -e fast	Unit + integration	~30s	Combined, no e2e/functional/external
tox -e ollama	Live LLM tests	Variable	Requires running Ollama instance
tox -e lint	Code quality gate	Fast	ruff check + format check + inline CSS check
tox -e format	Auto-format	Fast	ruff fix + ruff format
tox -e typecheck	Type checking	Medium	mypy static analysis
tox -e ci	Full CI suite	Slow	Coverage + JUnit XML output
tox -e pre-push	Pre-push gate	Medium	lint + full CI (mirrors Gitea Actions)
tox -e benchmark	Performance regression	Variable	Agent performance benchmarks

Test Markers

Tests are organized with pytest markers defined in pyproject.toml:

• unit - Fast unit tests, no I/O, no external dependencies
• integration - May use SQLite databases, file I/O
• functional - Real HTTP requests against test servers
• e2e - Full system end-to-end tests
• dashboard - Dashboard route tests
• slow - Tests taking >1 second
• ollama - Requires live Ollama instance
• docker - Requires Docker
• selenium - Requires browser automation
• external_api - Requires external API access
• skip_ci - Skipped in CI

Mark your tests in the test file:

import pytest

@pytest.mark.unit
def test_something():
    assert True

@pytest.mark.integration
def test_with_database():
    # Uses SQLite or file I/O
    pass

Test Directory Structure

tests/
  unit/           - Fast unit tests
  integration/    - Integration tests (SQLite, file I/O)
  functional/     - Real HTTP tests
  e2e/            - End-to-end system tests
  conftest.py     - Shared fixtures

Writing New Tests

1. Place your test in the appropriate directory (tests/unit/, tests/integration/, etc.)
2. Use the correct marker (@pytest.mark.unit, @pytest.mark.integration, etc.)
3. Test file names must start with test_
4. Use fixtures from conftest.py for common setup

Example

# tests/unit/test_my_feature.py
import pytest

@pytest.mark.unit
class TestMyFeature:
    def test_basic_behavior(self):
        result = my_function("input")
        assert result == "expected"

    def test_edge_case(self):
        with pytest.raises(ValueError):
            my_function(None)

Environment Variables

The test suite sets these automatically via tox:

• TIMMY_TEST_MODE=1 - Enables test mode in the application
• TIMMY_DISABLE_CSRF=1 - Disables CSRF protection for test requests
• TIMMY_SKIP_EMBEDDINGS=1 - Skips embedding generation (slow)

Git Hooks

Pre-commit and pre-push hooks run tests automatically:

• Pre-commit: tox -e format then tox -e unit
• Pre-push: tox -e pre-push (lint + full CI)

Never use --no-verify on commits or pushes.

CI Pipeline

Gitea Actions runs on every push and PR:

1. Lint: tox -e lint - code quality gate
2. Unit tests: tox -e unit - fast feedback
3. Integration tests: tox -e integration
4. Coverage: tox -e ci - generates coverage.xml

The CI fails if:

• Any lint check fails
• Any test fails
• Coverage drops below the threshold (see pyproject.toml [tool.coverage.report])

Troubleshooting

Tests timeout: Increase timeout with pytest --timeout=120 or check for hanging network calls.

Import errors: Run pip install -e ".[dev]" to ensure all dependencies are installed.

Ollama tests fail: Ensure Ollama is running at the configured OLLAMA_URL.

Flaky tests: Mark with @pytest.mark.slow if genuinely slow, or file an issue if intermittently failing.