feat: implement TurboQuant upstream watch monitoring system
All checks were successful
Smoke Test / smoke (pull_request) Successful in 28s
All checks were successful
Smoke Test / smoke (pull_request) Successful in 28s
- Add scripts/upstream_watch.py for monitoring upstream repositories - Add .github/workflows/upstream-watch.yml for weekly automated monitoring - Add docs/upstream-watch.md for documentation - Add scripts/run_upstream_watch.sh for easy execution - Add scripts/test_upstream_watch.py for testing Addresses issue #15: [P4] Upstream llama.cpp / Ollama TurboQuant watch Features: 1. Monitor llama.cpp, Ollama, and ggml repositories 2. Search for TurboQuant/PolarQuant/QJL keywords 3. Check issues, PRs, and release notes 4. Generate text and JSON reports 5. Weekly GitHub Action for continuous monitoring 6. Automated issue creation when findings detected Usage: - Run monitor: python3 scripts/upstream_watch.py --days 30 - JSON output: python3 scripts/upstream_watch.py --format json - Weekly monitoring: GitHub Action runs every Monday at 9:00 AM UTC When upstream lands: 1. Detection: Monitor will detect mentions 2. Evaluation: Compare upstream vs fork 3. Decision: Migrate if upstream is better Closes #15
This commit is contained in:
119
.github/workflows/upstream-watch.yml
vendored
Normal file
119
.github/workflows/upstream-watch.yml
vendored
Normal file
@@ -0,0 +1,119 @@
|
||||
# .github/workflows/upstream-watch.yml
|
||||
# Weekly TurboQuant upstream monitoring
|
||||
|
||||
name: TurboQuant Upstream Watch
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Run every Monday at 9:00 AM UTC
|
||||
- cron: '0 9 * * 1'
|
||||
workflow_dispatch: # Allow manual triggers
|
||||
inputs:
|
||||
days:
|
||||
description: 'Number of days to scan'
|
||||
required: false
|
||||
default: '30'
|
||||
|
||||
jobs:
|
||||
upstream-watch:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
# No additional dependencies needed
|
||||
|
||||
- name: Run upstream watch
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
# Get days from input or use default
|
||||
DAYS="${{ github.event.inputs.days || '30' }}"
|
||||
|
||||
# Run the monitor
|
||||
python scripts/upstream_watch.py --days "$DAYS" --format json --output upstream-report.json
|
||||
|
||||
# Also generate text report
|
||||
python scripts/upstream_watch.py --days "$DAYS" --format text --output upstream-report.md
|
||||
|
||||
# Check if there are findings
|
||||
FINDINGS=$(python -c "import json; data=json.load(open('upstream-report.json')); print(data['total_found'])")
|
||||
|
||||
if [ "$FINDINGS" -gt 0 ]; then
|
||||
echo "⚠️ Found $FINDINGS TurboQuant mentions in upstream repositories"
|
||||
echo "::warning::Found $FINDINGS TurboQuant mentions in upstream repositories"
|
||||
else
|
||||
echo "✅ No TurboQuant mentions found in upstream repositories"
|
||||
fi
|
||||
|
||||
- name: Upload reports
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: upstream-reports
|
||||
path: |
|
||||
upstream-report.json
|
||||
upstream-report.md
|
||||
retention-days: 30
|
||||
|
||||
- name: Create issue if findings
|
||||
if: ${{ hashFiles('upstream-report.json') != '' }}
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
const report = JSON.parse(fs.readFileSync('upstream-report.json', 'utf8'));
|
||||
|
||||
if (report.total_found > 0) {
|
||||
const issueBody = `## TurboQuant Upstream Findings
|
||||
|
||||
**Scan Date:** ${report.scan_date}
|
||||
**Days Scanned:** ${report.days_scanned}
|
||||
**Total Findings:** ${report.total_found}
|
||||
|
||||
### llama.cpp Mentions
|
||||
${report.llama_cpp_results.length > 0 ?
|
||||
report.llama_cpp_results.map(r => `- [${r.type.toUpperCase()}] ${r.repo}#${r.number}: ${r.title}\n URL: ${r.url}`).join('\n') :
|
||||
'No mentions found'}
|
||||
|
||||
### Ollama Mentions
|
||||
${report.ollama_results.length > 0 ?
|
||||
report.ollama_results.map(r => `- [${r.type.toUpperCase()}] ${r.repo}#${r.number}: ${r.title}\n URL: ${r.url}`).join('\n') :
|
||||
'No mentions found'}
|
||||
|
||||
### Ollama Releases
|
||||
${report.ollama_releases.length > 0 ?
|
||||
report.ollama_releases.map(r => `- ${r.version}: ${r.name}\n URL: ${r.url}\n Keywords: ${r.keywords.join(', ')}`).join('\n') :
|
||||
'No releases with TurboQuant mentions'}
|
||||
|
||||
### Recommendation
|
||||
${report.total_found > 0 ?
|
||||
'⚠️ Found TurboQuant mentions in upstream. Evaluate whether to migrate to upstream or continue using fork.' :
|
||||
'✅ No TurboQuant mentions found. Continue using fork.'}
|
||||
|
||||
---
|
||||
*Generated by upstream-watch workflow*`;
|
||||
|
||||
await github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title: `TurboQuant Upstream Findings: ${report.total_found} mentions found`,
|
||||
body: issueBody,
|
||||
labels: ['upstream-watch', 'turboquant']
|
||||
});
|
||||
}
|
||||
|
||||
- name: Commit reports
|
||||
run: |
|
||||
git config --local user.email "action@github.com"
|
||||
git config --local user.name "GitHub Action"
|
||||
git add upstream-report.json upstream-report.md
|
||||
git commit -m "docs: update upstream watch reports [skip ci]" || echo "No changes to commit"
|
||||
git push || echo "Push failed (might be on protected branch)"
|
||||
Reference in New Issue
Block a user