🛣️ LANYX

Live, on-device, top-down lane mapping from a dash-mounted phone

Point a phone at the road. LANYX detects lane markings in the camera feed, lifts them onto the ground plane using ARCore's tracked pose, and accumulates a growing top-down lane map in real time, fully on-device. Then it vectorizes that map into real lane geometry and exports it to GIS and HD-map formats.

---

✨ Highlights

	Feature
🎥	Real-time perception on the GL thread: ARCore owns the camera, detection runs on a single latest-frame-wins worker, never blocking render.
🧠	Pluggable detectors: a pure-Kotlin classical detector (HSV + Otsu + connected-component shape filtering, no OpenCV) and a LiteRT / TFLite segmentation path with a GPU to NNAPI to CPU acceleration chain.
🗺️	Growing top-down map: marking pixels are lifted to ground coordinates via a flat-ground homography and accumulated into a memory-bounded density grid.
🎨	Class-aware: white vs yellow markings tracked per cell and rendered cyan vs amber, live on the camera.
📐	Lane vectorization: density grid to centerlines (morphological close, Zhang-Suen thinning, skeleton graph trace, Douglas-Peucker) with per-lane color, width, and dashed/solid.
🌍	Standard exports: GeoJSON, OpenDRIVE, Lanelet2, KML, JSON, PNG, plus an ML dataset exporter.
🎬	Record, replay, and tune: capture raw clips, replay them through the exact live pipeline, and sweep parameters to auto-pick the best settings offline.
📍	GPS-track georeferencing: a rigid 2D fit of the local path to the GPS track gives real lat/lon with a known heading.
⚙️	Production-grade plumbing: lifecycle-safe threading, GZIP session persistence with crash-safe atomic writes, thermal-aware throttling, permission recovery, and a foreground-service mapping session.

---

🧭 Architecture

flowchart LR
    CAM([📷 Camera]) --> AR[ARCore VIO<br/>pose + YUV frames]
    AR --> GL{{GL render thread}}
    GL -->|latest-frame-wins| WK[[Inference worker]]
    WK --> DET[Lane detector<br/>Classical / LiteRT]
    DET --> LIFT[Pixel to ground<br/>flat-ground homography]
    LIFT --> MAP[(Top-down<br/>density grid)]
    GL --> OVL[Segmentation overlay]
    MAP --> VEC[Vectorizer<br/>thin + trace + simplify]
    VEC --> EXP[[Exports:<br/>GeoJSON / OpenDRIVE /<br/>Lanelet2 / KML / dataset]]
    GPS([🛰️ GPS]) --> GEO[Georeferencer]
    MAP --> GEO --> EXP
    MAP --> PERSIST[(GZIP session<br/>autosave + resume)]

Loading

Threading model (never block the camera):

GL render thread        Inference worker          UI tickers
----------------        -----------------         -----------
session.update()        take latest frame         stats  ~5 Hz
draw camera bg          YUV -> RGB                 map invalidate ~30 Hz
draw mask overlay       detect -> lift -> map      cache rebuild (bg)
submit frame  ───────▶  publish mask  ───────────▶ autosave ~20 s
onArFrame (pose,                                   foreground notification
 ground, speed, geo)

Module layout (app/src/main/java/com/lanyx/mapper/):

ar/         ARCore session lifecycle, install flow, frame geometry
gl/         GLSurfaceView renderer, camera background, mask + dim overlays
detect/     LaneDetector interface, Classical (HSV+Otsu+CCL), LiteRT, YUV->RGB
geometry/   ImageTransform, Lifter (ray to ground), GroundPlane, intrinsics
pipeline/   CaptureCoordinator, FrameProcessor, InferenceWorker, MaskBus, buffers
map/        LaneMap density grid, GridCell, Trajectory, CellKey
vector/     LaneVectorizer (centerlines + color/width/dashed)
geo/        GeoReferencer (rigid GPS-track fit), GeoTransform
export/     GeoJSON, OpenDRIVE, Lanelet2, KML, ML dataset, map JSON/PNG
record/     ClipRecorder, ClipReplayer, ClipCodec
tune/       ParameterSweep
location/   GpsManager, GpsFix
persistence/ SessionStore, SessionCodec (GZIP, versioned, back-compatible)
service/    CaptureService (foreground)
config/     AppConfig, SettingsRepository (DataStore)
ui/         MainActivity, SettingsActivity, MapView, StatsBarView

---

🚀 Quick start

Requirements: an ARCore-supported Android device (8.0+), OpenGL ES 3.0, JDK 17, Android SDK (API 36).

git clone https://github.com/Sherin-SEF-AI/LANYX.git
cd LANYX

# point local.properties at your SDK (created automatically by Android Studio),
# or:  echo "sdk.dir=$ANDROID_HOME" > local.properties

./gradlew :app:assembleDebug
./gradlew :app:installDebug      # to a connected device
./gradlew :app:testDebugUnitTest # 63 unit tests

Mount the phone on the dash facing forward, grant Camera + Location, tap START, and drive.

---

🎛️ Controls

Control	Action
START / PAUSE	Begin or pause accumulating into the map
CLEAR	Reset the map (and saved session)
FOLLOW	Keep the ego vehicle centered in the top-down view
EXPORT	Write the map to JSON, PNG, GeoJSON, OpenDRIVE, KML, Lanelet2 and share
CFG	Settings: mount height, lift range, detector, ROI, grid size, auto start/stop, overlay, and more
REC / REPLAY / SWEEP	Record a raw clip, replay it through the pipeline, or sweep tuning parameters
SEG	Cycle the segmentation overlay: hidden, visible, fullscreen

Auto start/stop can begin and pause mapping by vehicle speed, with manual override.

---

🧩 Lane detection

Classical (default, works out of the box, no model): white and yellow gating with an Otsu-adaptive brightness floor and brightness adaptation for day/night, followed by connected-component shape filtering that keeps thin, elongated, lane-sized segments and rejects walls, sky, and glare.

LiteRT / TFLite (model-driven): a semantic-segmentation model loaded from on-device storage or assets, accelerated GPU to NNAPI to CPU. A small demo model ships in assets/lane_seg.tflite so the path runs out of the box; drop in a trained model for real lane detection:

adb push lane_seg.tflite /sdcard/Android/data/com.lanyx.mapper/files/lane_seg.tflite

I/O contract: input float32 [1, H, W, 3] normalized to [-1, 1], output float32 [1, H, W, C] (C == 1 sigmoid, or C >= 2 argmax with class 0 = background). The app reads the model's input dimensions and adapts.

---

🌍 Outputs and exports

Format	File	Contents
🗺️ GeoJSON	.geojson	Lane LineStrings with color, widthM, dashed, lengthM, plus markingCount. WGS84 lat/lon when georeferenced, else local metres.
🛣️ OpenDRIVE	.xodr	ASAM 1.7 roads with planView geometry and centerline roadMark color/type/width (local frame).
🧷 Lanelet2	.osm	Lane-boundary linestrings tagged type / subtype (solid or dashed) / color.
🌎 KML	.kml	Color-styled lanes for Google Earth and Maps.
🧮 Map JSON	.json	Raw cells, trajectory, and reproducible capture settings.
🖼️ PNG	.png	Snapshot of the top-down view.
🧪 ML dataset	dataset_<clip>/	frames/*.jpg + labels/*.png (per-pixel class) + manifest.jsonl (pose, intrinsics, timestamp) from a recorded clip, for training a real lane model offline.

---

🎬 Record, replay, and tune

LANYX records the raw stream (camera frames + ARCore pose + GPS) to a clip, then replays it through the exact same detect to lift to accumulate path used live. That makes offline iteration possible:

1. Record a drive once.
2. Replay it after changing settings to see the effect, no re-driving.
3. Sweep a grid of detector and lifting parameters across the clip; LANYX scores each result by how lane-like the output is and proposes the best configuration to apply.

---

📍 Georeferencing

Markings are accumulated in ARCore's local metric frame. When GPS fixes are available during a drive, LANYX captures local-pose to GPS correspondences and fits a rigid 2D transform (rotation + translation, closed-form Procrustes) to place the map in real WGS84 coordinates with a known heading. Quality gates (minimum baseline, correspondence count, and residual) decide whether the heading is trusted; otherwise exports fall back to a documented local frame.

---

🧪 Tests

63 JVM unit tests cover the pure logic: the session codec (including legacy back-compatibility), the density grid and class accumulation, the vectorizer (color, width, dashed, skeleton tracing), the georeferencer (rotation recovery, degenerate rejection), the export formats, the classical detector gating, the lifter, and the capture buffers.

./gradlew :app:testDebugUnitTest

---

📦 Tech stack

Kotlin + Coroutines | ARCore 1.54 | LiteRT (TensorFlow Lite) GPU / NNAPI / CPU | OpenGL ES 3.0 | Jetpack DataStore | Play Services Location | Material 3 | Gradle (AGP 9, JDK 17, minSdk 24, target 36)

---

⚠️ Scope

LANYX is a lane-perception and lane-mapping / HD-map data-export tool and research platform. It is not a self-driving stack: there is no object or sign perception, no centimeter-level localization, no planning or control, no sensor redundancy, and no functional-safety guarantee. Treat its output as a mapping and research artifact (a lane-departure warning could be built on it), not a vehicle control input. The bundled TFLite model is a deterministic demo, not a trained lane network.

🔐 Privacy

Recorded clips and exported datasets contain camera imagery and precise location. Handle them as personal and location data: get consent, set retention, and strip location before sharing if needed.

🛠️ Roadmap

• Trained lane-segmentation model and on-road validation
• Mount pitch/roll calibration for better lifting accuracy
• Grouping boundaries into drivable lanelet and OpenDRIVE lane topology
• ARCore Geospatial (VPS) for drift-corrected global anchoring
• Change detection between repeated drives

---

Built by sherin joseph roy · sherin.joseph2217@gmail.com

If LANYX is useful to you, consider starring the repo ⭐