[loop-generated] [refactor] Split research.py — 528 lines, research coordination system #1405

Closed
opened 2026-03-24 12:53:36 +00:00 by Timmy · 4 comments
Owner

Problem

src/timmy/research.py is 528 lines with multiple responsibilities:

  • Research task coordination
  • Data source orchestration
  • Analysis pipeline management
  • Result aggregation and reporting
  • Research session state management

Proposed Split

  1. Extract task coordination into src/timmy/research/coordinator.py
  2. Extract data sources into src/timmy/research/sources.py
  3. Extract analysis pipeline into src/timmy/research/analysis.py
  4. Extract reporting into src/timmy/research/reporting.py
  5. Keep research.py as lightweight orchestrator

Benefits

  • Better separation of research concerns
  • Testable components in isolation
  • Reusable data source modules
  • Cleaner research pipeline architecture

Acceptance Criteria

  • No module exceeds 200 lines after split
  • All existing research functionality preserved
  • All tests pass (tox -e unit)
  • Research pipeline works end-to-end
  • Clean module boundaries with minimal inter-dependencies

Files

  • src/timmy/research.py (primary, 528 lines)

Lines of code is a liability. Delete as much as you create.

## Problem `src/timmy/research.py` is 528 lines with multiple responsibilities: - Research task coordination - Data source orchestration - Analysis pipeline management - Result aggregation and reporting - Research session state management ## Proposed Split 1. Extract task coordination into `src/timmy/research/coordinator.py` 2. Extract data sources into `src/timmy/research/sources.py` 3. Extract analysis pipeline into `src/timmy/research/analysis.py` 4. Extract reporting into `src/timmy/research/reporting.py` 5. Keep `research.py` as lightweight orchestrator ## Benefits - Better separation of research concerns - Testable components in isolation - Reusable data source modules - Cleaner research pipeline architecture ## Acceptance Criteria - [ ] No module exceeds 200 lines after split - [ ] All existing research functionality preserved - [ ] All tests pass (`tox -e unit`) - [ ] Research pipeline works end-to-end - [ ] Clean module boundaries with minimal inter-dependencies ## Files - `src/timmy/research.py` (primary, 528 lines) Lines of code is a liability. Delete as much as you create.
Author
Owner

Implementation Instructions

Step-by-Step Implementation:

  1. Create package structure:

    src/timmy/research/
    ├── __init__.py
    ├── coordinator.py (task coordination)
    ├── sources.py (data source management)
    ├── analysis.py (analysis pipeline)
    └── reporting.py (result reporting)
    
  2. Components to Extract:

    • coordinator.py: Research task orchestration, workflow management, progress tracking
    • sources.py: Data source connectors, API integrations, web scraping, document processing
    • analysis.py: Analysis pipelines, data processing, summarization, insights extraction
    • reporting.py: Result formatting, report generation, visualization data prep
  3. Keep in research.py (orchestrator):

    • Main conduct_research() function
    • High-level workflow coordination
    • Error handling and logging
    • Public API surface
  4. Key Migration Points:

    • Preserve all existing research functionality
    • Maintain current API contracts
    • Ensure clean data flow between components
    • No loss of configuration or state
  5. Testing:

    tox -e unit
    # Test research pipeline end-to-end
    # Verify all data sources still work
    

Goal: Transform monolithic research module into clean, testable components while preserving all functionality.

## Implementation Instructions ### Step-by-Step Implementation: 1. **Create package structure:** ``` src/timmy/research/ ├── __init__.py ├── coordinator.py (task coordination) ├── sources.py (data source management) ├── analysis.py (analysis pipeline) └── reporting.py (result reporting) ``` 2. **Components to Extract:** - **coordinator.py**: Research task orchestration, workflow management, progress tracking - **sources.py**: Data source connectors, API integrations, web scraping, document processing - **analysis.py**: Analysis pipelines, data processing, summarization, insights extraction - **reporting.py**: Result formatting, report generation, visualization data prep 3. **Keep in research.py (orchestrator):** - Main `conduct_research()` function - High-level workflow coordination - Error handling and logging - Public API surface 4. **Key Migration Points:** - Preserve all existing research functionality - Maintain current API contracts - Ensure clean data flow between components - No loss of configuration or state 5. **Testing:** ```bash tox -e unit # Test research pipeline end-to-end # Verify all data sources still work ``` **Goal**: Transform monolithic research module into clean, testable components while preserving all functionality.
kimi was assigned by Timmy 2026-03-24 12:54:48 +00:00
kimi was unassigned by Timmy 2026-03-24 18:26:16 +00:00
Author
Owner

Kimi Task: Split research.py

Branch:

What to do:

  1. Create package with
  2. Extract to :
    • async function (main entry point)
    • async function
    • and functions
    • async function
  3. Extract to :
    • async function
    • async function
  4. Extract to :
    • function
    • function
    • function
  5. Extract to :
    • async function
    • dataclass
  6. Extract to :
    • function
    • function
  7. Keep as thin re-export module for backward compat: etc.

Note: research.py is NOT imported directly by other src/ files, so the re-export shim is just for safety.

Verification: ROOT: No loadable tox.ini or setup.cfg or pyproject.toml or tox.toml found, assuming empty tox.ini at /Users/apayne
ROOT: HandledError| provided environments not found in configuration file:
unit must pass. No new tests needed.

**Kimi Task: Split research.py** Branch: **What to do:** 1. Create package with 2. Extract to : - async function (main entry point) - async function - and functions - async function 3. Extract to : - async function - async function 4. Extract to : - function - function - function 5. Extract to : - async function - dataclass 6. Extract to : - function - function 7. Keep as thin re-export module for backward compat: etc. **Note:** research.py is NOT imported directly by other src/ files, so the re-export shim is just for safety. **Verification:** ROOT: No loadable tox.ini or setup.cfg or pyproject.toml or tox.toml found, assuming empty tox.ini at /Users/apayne ROOT: HandledError| provided environments not found in configuration file: unit must pass. No new tests needed.
kimi was assigned by Timmy 2026-03-24 18:43:38 +00:00
Author
Owner

@kimi Instructions:

Files: src/timmy/research/coordinator.py, src/timmy/research.py, src/timmy/research/sources.py

Split research.py (528 lines) into focused modules:

  1. Extract coordination logic to coordinator.py
  2. Extract source-specific code to sources.py
  3. Keep research.py as thin facade

Verify: tox -e unit passes, tox -e lint clean

@kimi Instructions: Files: src/timmy/research/coordinator.py, src/timmy/research.py, src/timmy/research/sources.py Split research.py (528 lines) into focused modules: 1. Extract coordination logic to coordinator.py 2. Extract source-specific code to sources.py 3. Keep research.py as thin facade Verify: tox -e unit passes, tox -e lint clean
kimi was unassigned by Timmy 2026-03-24 19:18:21 +00:00
Author
Owner

Instructions for Kimi

Goal: Split src/timmy/research.py (528 lines) into focused modules.

Files to create/modify:

  1. src/timmy/research/coordinator.py - Extract research coordination logic, task orchestration
  2. src/timmy/research/sources.py - Extract source management, API integrations for research
  3. src/timmy/research.py - Keep as thin facade/entry point, import from new modules

Steps:

  1. Read src/timmy/research.py and identify logical groupings
  2. Create src/timmy/research/ directory with init.py
  3. Create coordinator.py and sources.py with extracted code
  4. Update research.py to re-export from submodules
  5. Run tox -e format then tox -e unit - all tests must pass
  6. Run tox -e lint - must be clean

Acceptance criteria:

  • research.py reduced to <100 lines (facade only)
  • coordinator.py and sources.py contain the extracted logic
  • All existing tests pass unchanged
  • No circular imports
  • tox -e lint clean
## Instructions for Kimi **Goal:** Split src/timmy/research.py (528 lines) into focused modules. **Files to create/modify:** 1. src/timmy/research/coordinator.py - Extract research coordination logic, task orchestration 2. src/timmy/research/sources.py - Extract source management, API integrations for research 3. src/timmy/research.py - Keep as thin facade/entry point, import from new modules **Steps:** 1. Read src/timmy/research.py and identify logical groupings 2. Create src/timmy/research/ directory with __init__.py 3. Create coordinator.py and sources.py with extracted code 4. Update research.py to re-export from submodules 5. Run tox -e format then tox -e unit - all tests must pass 6. Run tox -e lint - must be clean **Acceptance criteria:** - research.py reduced to <100 lines (facade only) - coordinator.py and sources.py contain the extracted logic - All existing tests pass unchanged - No circular imports - tox -e lint clean
kimi was assigned by Timmy 2026-03-24 19:36:22 +00:00
kimi was unassigned by Timmy 2026-03-24 19:41:21 +00:00
Timmy self-assigned this 2026-03-24 19:41:21 +00:00
Timmy closed this issue 2026-03-24 19:53:06 +00:00
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#1405