Add Project_notes.md — centralized status tracker for all side projects
This commit is contained in:
nightwing
136
Project_notes.md
Normal file
136
Project_notes.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Project Notes
|
||||
|
||||
*Maintained by Hermes — last updated February 2025*
|
||||
|
||||
---
|
||||
|
||||
## 1. Kandinsky (Multimodal Transformer)
|
||||
- **Repo:** https://github.com/samherring99/kandinsky
|
||||
- **Local path:** `~/Desktop/Projects/kandinsky`
|
||||
- **Description:** An anything-to-anything transformer combining text, image, and audio modalities. Trains on Pokemon BLIP captions paired with Gen 1 Pokemon audio cries. Uses audio tokenization adapted from nanoGPT.
|
||||
- **Status:** Early POC. Training code exists (`model.py`) and dataset creation (`create_dataset.py`) works. Audio heads are producing the same sound — unclear if it's a training issue or data issue.
|
||||
- **TODO:**
|
||||
- Debug why audio heads produce identical output
|
||||
- Investigate if model needs more training time
|
||||
- Design a data pipeline for better/more training data
|
||||
- General repo cleanup (requirements.txt, proper CLI, etc.)
|
||||
|
||||
---
|
||||
|
||||
## 2. NightwingGameSim (LLM → GameBoy ROM Generator)
|
||||
- **Repo:** https://github.com/samherring99/NightwingGameSim
|
||||
- **Local path:** `~/Desktop/Projects/NightwingGameSim`
|
||||
- **Description:** AI-powered pipeline that turns natural language prompts into playable GameBoy ROM files. Generates C code, compiles with GBDK, outputs `.gb` files. Supports Claude API, local Llama, and RAG backends.
|
||||
- **Status:** Functional — generation pipeline works end-to-end with Claude 4 system prompt. Has tests, docs, examples, and retry logic.
|
||||
- **TODO:**
|
||||
- Harden the repo, clean up structure
|
||||
- Build a better testing pipeline
|
||||
- Come up with better prompt ideas / examples
|
||||
|
||||
---
|
||||
|
||||
## 3. ContentBasedMIR (Music Information Retrieval)
|
||||
- **Repo:** https://github.com/samherring99/ContentBasedMIR
|
||||
- **Local path:** `~/Desktop/Projects/ContentBasedMIR`
|
||||
- **Description:** Music similarity analysis using Spotify API track data. Extracts 54 audio features per song and visualizes similarity matrices for music recommendation.
|
||||
- **Status:** Early stage. Can download Spotify track analysis data and plot similarity matrices. Needs significant expansion.
|
||||
- **TODO:**
|
||||
- Expand analysis pipeline with more features
|
||||
- Integrate with text message data for personalized recommendations
|
||||
- Build out visualization and exploration tools
|
||||
- General modernization (dependencies, structure)
|
||||
|
||||
---
|
||||
|
||||
## 4. MessageRetrieval (iMessage RAG/SQL)
|
||||
- **Repo:** https://github.com/samherring99/MessageRetrieval
|
||||
- **Local path:** `~/Desktop/Projects/MessageRetrieval`
|
||||
- **Description:** Natural language querying over iMessage data using SQL generation (text2SQL) instead of vector embeddings. Uses LLM-as-Judge pattern for scoring and ranking retrieved messages.
|
||||
- **Status:** Has initial text2SQL pipeline and summarization tool. Recently worked on with Claude Code. Needs testing.
|
||||
- **TODO:**
|
||||
- Test out the recent Claude Code work
|
||||
- Build "iMessage Jarvis" — answer questions about texts
|
||||
- Improve SQL generation prompts and accuracy
|
||||
- Better error handling and UX
|
||||
|
||||
---
|
||||
|
||||
## 5. Grailed Embedding Search
|
||||
- **Repo:** https://github.com/samherring99/grailed-embedding-search
|
||||
- **Local path:** `~/Desktop/Projects/grailed-embedding-search`
|
||||
- **Description:** Embedding-based semantic search over Grailed fashion listings. Uses vector similarity to find related items.
|
||||
- **Status:** Very early — has a basic similarity search implementation. Previously had a more complex version that's being reworked.
|
||||
- **TODO:**
|
||||
- Build out the search pipeline
|
||||
- Add scraping/indexing for listings
|
||||
- Improve embedding approach
|
||||
- Add UI or CLI for exploring results
|
||||
|
||||
---
|
||||
|
||||
## 6. NightwingNBA (Sports Analytics)
|
||||
- **Repo:** https://github.com/samherring99/NightwingNBA
|
||||
- **Local path:** `~/Desktop/Projects/NightwingNBA`
|
||||
- **Description:** NBA game prediction system. Builds a database of game data, trains a PyTorch model, and makes daily predictions. Has full pipeline: build DB → write data → train → predict.
|
||||
- **Status:** Functional pipeline exists. Has database building, training, prediction, and daily update scripts.
|
||||
- **TODO:**
|
||||
- Explore and potentially revive
|
||||
- Update data sources if stale
|
||||
- Improve model accuracy
|
||||
- Add visualization/reporting
|
||||
|
||||
---
|
||||
|
||||
## 7. Stable Audio Sample Explorer
|
||||
- **Repo:** https://github.com/samherring99/stable-audio-sample-explorer
|
||||
- **Local path:** `~/Desktop/Projects/stable-audio-sample-explorer`
|
||||
- **Description:** Tool for exploring audio samples generated by Stable Audio.
|
||||
- **Status:** 🪦 **Dead** — no active work needed per Sam.
|
||||
|
||||
---
|
||||
|
||||
## 8. NightwingArt (Art Tools)
|
||||
- **Repo:** https://github.com/samherring99/NightwingArt
|
||||
- **Local path:** `~/Desktop/Projects/NightwingArt`
|
||||
- **Description:** Collection of art tooling scripts — video editing, clip splicing with beat matching, damage effects, and general image manipulation.
|
||||
- **Status:** Maintenance mode. Tools exist for various effects. Work happens as-needed.
|
||||
- **TODO:**
|
||||
- Add tools as needed for new art projects
|
||||
|
||||
---
|
||||
|
||||
## 9. Claude-based VST Building ⚠️ *Needs new repo*
|
||||
- **Description:** Generate VST audio plugins for DAWs from English language prompts. LLM-powered audio plugin creation.
|
||||
- **Status:** Concept only — no repo exists yet.
|
||||
- **TODO:**
|
||||
- Create repo
|
||||
- Research VST SDK / JUCE framework
|
||||
- Design prompt → code → compile pipeline
|
||||
|
||||
---
|
||||
|
||||
## 10. Government Auction Site Scraper ⚠️ *Needs new repo*
|
||||
- **Description:** Tool that monitors and scrapes government auction sites in San Francisco for deals.
|
||||
- **Status:** Concept only — no repo exists yet.
|
||||
- **TODO:**
|
||||
- Create repo
|
||||
- Research SF government auction sites and their structure
|
||||
- Build scraper + notification system
|
||||
|
||||
---
|
||||
|
||||
## Priority Assessment
|
||||
|
||||
| Project | Activity Level | Suggested Priority |
|
||||
|---------|---------------|-------------------|
|
||||
| NightwingGameSim | Active | 🔴 High |
|
||||
| MessageRetrieval | Active | 🔴 High |
|
||||
| Kandinsky | Active | 🟡 Medium |
|
||||
| ContentBasedMIR | Exploratory | 🟡 Medium |
|
||||
| Grailed Embedding Search | Early | 🟡 Medium |
|
||||
| NightwingNBA | Dormant | 🟢 Low |
|
||||
| NightwingArt | As-needed | 🟢 Low |
|
||||
| VST Builder | Concept | 🔵 Future |
|
||||
| Gov Auction Scraper | Concept | 🔵 Future |
|
||||
| Stable Audio Explorer | Dead | ⚫ None |
|
||||
|
||||
Reference in New Issue
Block a user