Isoxml Ab Line Extraction

Task Details
← Task Board

Task Description

# TASK: Extract A-B Guidance Lines from ISOXML Data

**Priority:** low
**Agent:** data_pipeline
**Filed:** 2026-02-18
**Filed by:** Doug (via Claude Code session)

---

## Goal

Extract A-B guidance lines from Doug's ISOXML/TaskData files (TAP SFTP). These are stored as GuidanceGroup (GGP) and GuidancePattern (GPN) elements in TASKDATA.XML. The field-audit tool needs them — there's already a placeholder card on the dashboard.

## Background

- ISOXML spec (ISO 11783-10) stores guidance as:
  - `GGP` (GuidanceGroup) — container, links to a field (PFD)
  - `GPN` (GuidancePattern) — the actual A-B line geometry with type, points, heading
  - `GST` (GuidanceShift) — optional offset from the pattern
- Pattern types: `1`=AB, `2`=A+heading, `3`=curve, `4`=pivot, `5`=spiral
- The existing `isoxml_parser.py` has `guidance_patterns: List[dict]` in the Field dataclass but **never populates it**
- The `isoxml_processor.py` does NOT extract guidance at all

## What to Do

### 1. Survey the raw data first

Scan all ISOXML ZIPs in TAP SFTP storage on the Pi for GGP/GPN elements. Find out:
- How many ZIPs contain guidance data?
- What pattern types are present (AB, A+, curve, etc.)?
- Are they associated with specific fields via partfield refs?
- Sample the XML structure of a few GPN elements

```bash
# Scan TAP data directories for guidance
find /data/ -name "TASKDATA.XML" -o -name "taskdata.xml" | head -20
```

Then for each TASKDATA.XML found, check for GPN/GGP:
```python
import xml.etree.ElementTree as ET
tree = ET.parse('TASKDATA.XML')
root = tree.getroot()
gpn_elements = root.findall('.//GPN') + root.findall('.//GuidancePattern')
ggp_elements = root.findall('.//GGP') + root.findall('.//GuidanceGroup')
print(f"GPN: {len(gpn_elements)}, GGP: {len(ggp_elements)}")
for gpn in gpn_elements[:5]:
    print(ET.tostring(gpn, encoding='unicode'))
```

### 2. Add GPN/GGP parsing to the ISOXML parser

In `ISOBUS11783/tools/isoxml_parser.py`, add parsing for GuidancePattern elements. Key attributes:
- `A` = designator (name)
- `B` = pattern type (1=AB, 2=A+heading, 3=curve, 4=pivot, 5=spiral)
- `C` = heading (degrees, for A+ type)
- `D` = GNSS method
- `E` = horizontal accuracy
- `F` = propagation direction
- Child `LSG` (LineString) elements contain the point coordinates (ASN → Point with C=north, D=east)

### 3. Extract guidance lines per field

Write a script or extend the existing processor to:
- Parse all GPN elements from each TASKDATA.XML
- Associate them with fields via GGP → PFD (partfield) references
- Output per-field guidance lines as GeoJSON LineStrings with properties:
  - `name` (from GPN designator)
  - `pattern_type` (AB, A+heading, curve, etc.)
  - `heading_degrees`
  - `source_file`
  - `field_name` (from associated PFD)

### 4. Save output for field-audit

Save results to a JSON/GeoJSON file that the field-audit tool can consume:
```
farmiq.ai/field-audit/data/customers/doug/guidance/ab_lines.json
```

Format:
```json
{
  "customer": "doug",
  "extracted_at": "2026-02-18T...",
  "total_lines": 45,
  "fields": {
    "G01": [
      {
        "name": "AB Line 1",
        "type": "AB",
        "heading_deg": 172.5,
        "geometry": { "type": "LineString", "coordinates": [[lon,lat],[lon,lat]] },
        "source_file": "TASKDATA_20250501.zip"
      }
    ]
  }
}
```

### 5. Commit and report

- Commit any code changes and the extracted data to GitHub
- Post a summary: how many A-B lines found, for how many fields, which pattern types

## DO NOT

- Do NOT modify any raw TAP SFTP data
- Do NOT guess at coordinate systems — ISOXML uses WGS84 with coordinates in degrees * 10^-7 (integers). Convert correctly.
- Do NOT skip errors silently — log them and keep going
- Do NOT create test/dummy data — only extract what's actually in the files

Job Queue (0)

No job queue entries for this task yet