# TASK: Reprocess All Doug's TAP/ISOXML Data (10 Years)
**Priority:** HIGH
**Agent:** data_pipeline
**Filed:** 2026-02-16
**Filed by:** Doug (via Claude Code session)
---
## Goal
Process ALL of Doug's ISOXML data from TAP SFTP — 10 years of seeding, spray, harvest, and fertilizer records. This is the foundational dataset for FarmIQ. Process everything, fix errors as you go, keep going until done.
## Steps
### 1. Check current state
```bash
python /data/Sandbox/AgentPi/scripts/tap_batch_process.py --customer doug --status
```
See what's already processed, what failed, what's new.
### 2. Run full reprocessing
```bash
python /data/Sandbox/AgentPi/scripts/tap_batch_process.py --customer doug --retry-all
```
### 3. Fix errors and keep going
When processing fails:
1. Read the error message carefully
2. Determine if it's a code bug or bad data
3. If code bug: fix the code in `/data/Sandbox/ISOBUS11783/tools/isoxml_processor.py` or `/data/Sandbox/FarmData_API_Integration/file_router.py`
4. If bad data: log it and skip — don't waste time on corrupt files
5. Re-run processing after fixes
### 4. Report results
Post a summary to the dashboard when done:
```bash
curl -X POST http://localhost:8080/api/tasks -H "Content-Type: application/json" \
-d '{"title":"TAP Reprocess Complete", "description":"SUMMARY HERE — files processed, errors fixed, fields with data", "category":"report", "priority":"medium"}'
```
## Output
- Processed data in: `/data/processed_data/doug/{field}/{crop_year}/`
- Each field-year should have: metadata.json, seeding/spray/harvest data, boundaries
## Key Context
- TAP SFTP: `tap-storage-sftp.topconagriculture.com` (user: weistfarms@farmtech.us)
- Doug has 10 Topcon devices, XD+ 755D is the main seeder
- Previous stats: 764 synced, 640 ok. Target: get as close to 764/764 as possible.
- ISOXML processor is at `/data/Sandbox/ISOBUS11783/tools/isoxml_processor.py`
## DO NOT
- Do NOT skip errors silently — log every failure with the filename and error
- Do NOT modify the TAP SFTP data — read only
- Do NOT process Blanchet's data in this task — Doug only