# Dae Jang (Dae-Young Jang)

> I build robots and the vision and learning systems that drive them. Right now that means humanoid teleoperation and robot learning at RLWRLD. Looking for a Winter 2027 co-op (Jan – Apr 2027) in robotics, computer vision or machine learning.

This file summarises the portfolio for recruiters and AI agents. Every number comes from the project's own repository, recording or validation log; projects without measured results say so.

- Education: BASc Mechatronics Engineering (Honours, Co-op), University of Waterloo, expected Apr 2029
- Email: djang@uwaterloo.ca
- GitHub: https://github.com/djang-c
- LinkedIn: https://linkedin.com/in/dae-young-jang
- Site: https://djang.dev/
- Résumé (JSON Résumé): https://djang.dev/resume.json

## Experience

### RLWRLD: Forward Deployed Engineer (Sep 2026 – present, Bay Area)

Teleoperating humanoid robots, then teaching them from the demonstrations.

- Built a ROS 2 teleoperation pipeline that fuses a Quest headset, Vive tracker, Manus glove and two camera streams to drive an RBY1 + XHand robot, including the driver nodes and IK arm and hand retargeting.
- Ran teleoperation demos for AWS client proofs of concept that led to 3 strategic partnerships.
- Fine-tuned RLDX1 and RRC on 4 hours / 1,000 episodes of teleop data, reaching above 90% task success.
- Stack: ROS 2, Python, IK, Quest, Manus, RBY1, XHand

### RocScience: Computer Vision Engineer (Jan 2026 – Apr 2026, Toronto, ON)

Replacing a 30-minute sieve test with a real-time model.

- Cut soil particle-size analysis from 30 minutes to real time, validated against sieve testing, with a CNN + ViT ensemble trained on 12K+ images.
- Built YOLOv8 + SAM detection and segmentation that automated binary mask generation across incompatible datasets.
- Raised soil classification accuracy by 60% on unlabelled data with a ViT + Felzenszwalb + HDBSCAN clustering pipeline.
- Stack: PyTorch, ViT, YOLOv8, SAM, MiDaS, HDBSCAN

### White Hat Consulting: Cybersecurity Engineer (Jun 2025 – Aug 2025, Mexico City, Mexico)

Automating the reporting so the team could do the security work.

- Saved 16 hrs/month with a Power BI app on a private server, using REST APIs, Power Query and DAX to get past import limits securely.
- Saved 8 hrs/week by automating client report generation with a Python pipeline that builds and delivers Word reports.
- Stack: Python, Power BI, DAX, KQL, REST APIs

## Projects

### WALDO

A pan-tilt camera that finds a person and turns to keep them in view, in real time.

- Context: Own project
- Contribution: My own project: I designed the mount and wrote the clearance check.
- Approach: A camera rides on a pan-tilt head: a 37 mm gearmotor pans and a DS3240 servo tilts. The device finds the person in the camera image and turns both axes to keep them in view, showing the live feed with the tracked person on its own small screen. The mount is 45 grouped CAD parameters on an Alif E8 dev-kit base, checked for interference across its whole range.
- Evidence: Follows a moving person in real time on the built device (43 s recording). No part collisions across the full range, pan 0–180° and tilt ±90° (CAD clearance check).
- Areas: Robotics, Computer vision, Embedded, CAD
- Languages: Python
- Stack: CAD (STEP), Python, Alif E8
- Computer vision: Person tracking on the device. The camera feed is processed on the device itself: it finds the person in each frame and turns pan and tilt to keep them in view, while its own small screen shows the live feed with the tracked person marked.
- Robotics: A two-axis pan-tilt head. Pan: a 37 mm 12 V gearmotor, 0–180°. Tilt: a DS3240 digital servo, ±90°. The mount is 45 grouped CAD parameters, so one change (a camera, a clearance) rebuilds every dependent part. A clearance check sweeps the full pan and tilt range for part collisions. The first run caught the camera grazing the yoke at ±90°; the redesigned mount clears it in every pose.
- Firmware & electronics: Embedded platform. The head stands on an Alif E8 dev-kit base, which runs the tracking and drives both axes; nothing runs on a laptop.
- Python: Parametric CadQuery models of the gearmotor and servo, exported to STEP for the Onshape assembly, with every dimension tagged to its datasheet source.
- Code: https://github.com/djang-c/Auto-Flashlight

### WreckLess (Oct 2026)

An always-on security swarm on one desk-side Dell GB10: it reads a new advisory, exploits your code, patches it and proves the fix, then waits for a human to approve.

- Context: Dell × NVIDIA AI Hackathon, San Francisco
- Approach: Five agents on local models handle each finding: Scout reads the advisory, Ralph writes and fires an exploit in a deny-all-egress sandbox, a plain-code Verifier re-runs it on two vulnerable copies and a control with a fresh nonce, Felix patches the root cause, and Conductor posts a fix card to Slack and opens a draft PR only after Approve. No agent grades its own work: a patch must pass held-out regression tests, the re-run exploit and the app's own tests.
- Evidence: 3rd of 71 teams at the Dell × NVIDIA AI Hackathon, San Francisco, for running everything locally on the GB10. 5/5 live replays of CVE-2024-21513 (LangChain eval of database values) with local models and the sandbox on: 47–80 s from advisory to validated patch. Felix's patch matched the fix upstream shipped (ast.literal_eval) without seeing it. Four more CVE replays pass all 3 checks with the reference exploit and patch; live-model runs not reported yet.
- Areas: Security, AI agents, Applied AI
- Languages: Python, Shell
- Stack: Python, vLLM, Qwen3-Coder, NVIDIA GB10, Slack
- Model architecture: Three local models, sized to one box. Scout and Ralph run Qwen3-Coder-30B-A3B; Felix runs Qwen3-Coder-Next 80B-A3B. Both are mixture-of-experts models with about 3B parameters active per token, so generation stays fast while all three (~63 GB) stay resident in the GB10's 128 GB unified memory. vLLM serves them on loopback only; cloud LLM APIs are blocked at the firewall and checked by a lockdown script. Any agent's model can be swapped live from the app (e.g. NVIDIA Nemotron).
- Systems: No agent grades its own work. The exploit must make the vulnerable code itself write a random nonce to a file; a printed "success" counts for nothing. A plain-code Verifier re-runs it on three look-alike copies in random order (two vulnerable, one control): it must hit both and miss the control. A patch passes only if held-out regression tests Felix never sees pass, the re-run exploit now fails, and the app's own tests still pass. Exploits run in an OpenShell sandbox with deny-all egress and a read-only root.
- Systems: Always on, with a human in the loop. A systemd daemon checks advisories every 10 minutes and works the queue one finding at a time. Slack talks to it through an airlock: two processes, an outbound-only Socket Mode connection (no open port), and typed, size-capped messages between them. Approve is bound to the exact commit, and only then is a draft PR opened.
- Python: The agents, orchestrator, nonce Verifier, Slack airlock, daemon and full-screen app.
- Shell: Sandbox, firewall lockdown and verification, and the demo launcher.
- Code: https://github.com/djang-c/WreckLess-Cyber-Security

### Core Segmentation (Apr 2026)

Turns drill-core tray photos into one continuous, depth-labelled strip with every crack flagged.

- Context: RocScience core vision · own repository
- Contribution: I built the pipeline in my own repository during my RocScience term.
- Approach: Gemini proposes a box for each core run, SAM 2 masks it, MiDaS depth isolates the tray separators, and the masked crops are stitched into one strip with depth, sample length and cracks marked.
- Evidence: 20+ pixel-accurate masks per image across 100+ core samples.
- Areas: Computer vision, Machine learning
- Languages: Python
- Stack: Python, SAM 2, Gemini, MiDaS, OpenCV
- Computer vision: From a tray photo to one strip. Gemini proposes one box per core run, as normalised 0–1000 coordinates that are clamped and mapped back to pixels; truncated JSON replies are salvaged instead of failing the run. SAM 2 turns each box into a pixel mask. Each mask is used as the alpha channel of a tight crop, and the crops are laid left to right in shelf order and centred on one transparent strip, so tray background never reaches the output. A depth interval is spread across the strip in proportion to each segment's pixel width, with labelled boundaries. A second Gemini pass finds cracks; their boxes are projected into strip coordinates and drawn on the depth strip.
- Model architecture: Why SAM 2 with box prompts. SAM 2 encodes the image once with its Hiera image encoder; each box then goes through the light prompt encoder and mask decoder. One expensive encoding serves every core run in the tray, and the box prompt pins each mask to one run. The pipeline uses the small Hiera-S checkpoint (SAM 2.1).
- Python: An installable package: CLI pipeline, SAM 2 wrapper, strip stitching, depth layout and rendering, a Streamlit app, and tests for box normalisation.
- Code: private repository, available on request

### WATO ASD Robot (Sep 2025)

Point-to-point navigation for a simulated differential-drive robot: LiDAR costmap, A* planner and pure-pursuit control in a Dockerised ROS 2 and Gazebo stack.

- Context: WATonomous ASD project
- Evidence: 100% collision-free across the simulated test scenarios.
- Areas: Robotics, Path planning, Simulation
- Languages: C++
- Stack: C++, ROS 2, Gazebo, Docker, Foxglove
- Robotics: A full navigation stack, one ROS 2 node per job. Costmap: each LiDAR scan becomes a 300 × 300 occupancy grid at 0.1 m per cell; every hit is inflated over 1.5 m with cost falling off linearly with distance. Map memory: local costmaps are rotated by the robot's yaw and fused into a world-fixed map, only after the robot has moved far enough to add new information. Planner: A* with a Euclidean heuristic; a start or goal that lands inside an obstacle is moved to the nearest free cell within three cells. Control: pure pursuit at 10 Hz with a 1 m lookahead and curvature κ = 2y / L², angular speed clamped to 1.5 rad/s and a 0.2 m goal tolerance.
- C++: The costmap, map-memory, A* planner and pure-pursuit controller nodes (rclcpp).
- Code: https://github.com/djang-c/wato-asdrobot

### Trash Can Robot (Sep 2026 – Oct 2026)

An ESP32-S3 robot that carries a trash can: the camera streams to a ROS 2 stack for inference and control, and the firmware drives two 12 V gearmotors.

- Context: ESP32-S3 + ROS 2 build
- Evidence: No results reported yet.
- Areas: Robotics, Embedded, Computer vision
- Languages: C++, Python, OpenSCAD
- Stack: C++, ESP32, ROS 2, OpenSCAD, Python
- Robotics: ROS 2 on the laptop. camera_node decodes the MJPEG stream with OpenCV and publishes raw and compressed image topics for inference. motor_node turns velocity commands into UDP motor packets, re-sent at 20 Hz. safety_monitor clamps speeds and publishes zero if commands go stale (0.5 s) or the ESP32's heartbeat is lost (3 s). Foxglove for live visualisation.
- Firmware & electronics: ESP32-S3 firmware that fails safe. One FreeRTOS task owns the motor pins. Commands arrive over UDP, are guarded by a spinlock, and expire after 300 ms, so a dropped link or a dead laptop stops the robot instead of leaving it driving. 5 kHz, 8-bit LEDC PWM drives an L298N H-bridge for two 12 V gearmotors (differential drive). The OV3660 camera streams VGA MJPEG over HTTP from two frame buffers in PSRAM. Health is served on a separate port, because the stream blocks the first server's worker. A 10 s task watchdog, plus reboots after 60 s without WiFi or 10 straight camera failures; a 1 Hz UDP heartbeat; mDNS; its own access point if the network won't join. Pins avoid the S3's strapping, USB, UART0 and octal-PSRAM pins.
- Firmware & electronics: Power. 12 V from a USB-C PD trigger feeds the motor driver; an LM2596 buck, set to 5.0 V before the ESP32 is connected, powers the board, all on one shared ground.
- C++: ESP32-S3 firmware (Arduino on PlatformIO): motor task, camera, HTTP servers, UDP, watchdog, serial test commands.
- Python: The ROS 2 Humble nodes and drive scripts.
- OpenSCAD: The parametric chassis; every dimension lives in one parameters file.
- Code: https://github.com/ANonABento/trash-can-robot

### The Flight of the Buffalo (Oct 2026)

An evidence engine for how rare diseases connect, with a lab-automation step that is checked in MuJoCo before anything runs.

- Context: Hack-Nation Challenge 05
- Approach: Grounds each disease in HPO, MeSH and Europe PMC, links diseases through shared symptoms, genes and paper claims, and keeps an AI-extracted claim only if it quotes the paper word for word. A bounded MuJoCo simulation checks the liquid-handling step; the blocked run shown fails on purpose.
- Evidence: 52 source-quoted claims with DOIs. 2.48 s median for the five-call journey on the live app (10 runs).
- Areas: Applied AI, Lab automation, Simulation
- Languages: Python, TypeScript
- Stack: Python, TypeScript, MuJoCo, Docker, Cloud Run
- Robotics: The lab step is simulated before it runs. A MuJoCo gantry workcell: three slide joints (x, y, z) with set ranges carry a pipette that collides with a reagent tower between source and plate. A deterministic compiler turns an experiment spec into typed waypoints with a height for each operation, plus a ledger of liquid volumes and tips. Six checks run before simulation: spec schema, operation order, pipette capacity, source volume, tip availability and declared bounds. It rejects bad specs rather than clipping them. Fixtures cover a valid transfer, a blocked path, too little volume, no tips and an out-of-range move; the blocked run shown fails on purpose.
- Systems: The model proposes, code decides. A model reads the open-access full text and proposes typed claims with strict structured output. A claim is kept only if its quote appears word for word in the paper, names both entities, isn't negated, and both names resolve to MONDO, HGNC, HPO, GO or ChEBI IDs. Everything else is quarantined with its reason. A cross-paper hypothesis must cite stored claims from at least two papers, and is always labelled a hypothesis.
- Python: The evidence engine, paper ingestion and checks, API, and the robotics compiler and simulator.
- TypeScript: The explorer front end: graph, evidence drawer and simulation view.
- Code: https://github.com/djang-c/TFOTB

### Distance Scorer (Mar 2026 – Apr 2026)

Turns pixels-per-millimetre metadata into a calibrated apparent-distance score for soil photos, then trains a PyTorch regressor to predict it from the image alone.

- Context: Own project
- Evidence: Calibrated on 3,405 images spanning 8.8–136.2 PPM. Prediction accuracy not reported yet.
- Areas: Machine learning, Model training, Computer vision
- Languages: Python
- Stack: Python, PyTorch, SAM, pytest
- Computer vision: Building the dataset. Three soil and aggregate image sources (CloseRange, GeoM, SandSnap) are reformatted into one dataset; SAM segments the soil region of each image, and zoomed crops are generated with their pixels-per-millimetre adjusted.
- Model training: Training without leaking. Target: the natural log of pixels per millimetre, z-scored, so 34.5 PPM maps to 0 across 3,405 images. 70/15/15 split grouped by parent image, so a photo and its zoomed crops never land on both sides. Each sample's loss is weighted by label confidence (1.0 / 0.8 / 0.5). Brightness, contrast, blur and noise augmentation; flips stay off unless they suit the domain. Weight decay 0.01, LR 1e-3 decaying to 1e-6, early stopping after 10 epochs without improvement.
- Model architecture: A regressor that knows when it's unsure. A pretrained timm backbone (EfficientNet-B0; ConvNeXt-Tiny or ResNet-34 swap in) → pooling → MLP (256 hidden, dropout 0.2). Evidential head: four outputs parameterise a Normal–Inverse-Gamma (γ, ν, α, β), with softplus keeping ν, β > 0 and α > 1. γ is the prediction. Loss: the NIG negative log-likelihood (a Student-t predictive) plus a regulariser that penalises high evidence when |y − γ| is large, so confident wrong answers cost more. Evaluation reports error against uncertainty: MAE by uncertainty decile and a risk–coverage table for keeping only the most certain predictions.
- Python: PyTorch training and evaluation, the evidential loss, dataset building and SAM segmentation, scoring calibration, a synthetic benchmark, and pytest in CI.
- Code: private repository, available on request

### Baseline (Oct 2026)

Live deepfake-voice and scam-call detection that listens to both sides of a phone call.

- Context: Building from the Farm, SF Tech Week 2026
- Approach: Listens to both sides of the call and fuses speech detection, voice matching, a fine-tuned deepfake-audio detector, prosody against the speaker’s own baseline and LLM fraud-pattern checks into a live trust score, then signs the evidence into a verifiable call report.
- Evidence: Warns at 0:21 in the scripted demo call and holds the transfer.
- Areas: Audio AI, Machine learning, Security
- Languages: Python, TypeScript
- Stack: Python, TypeScript, Whisper, Ollama, ElevenLabs
- Model training: Fine-tuning a speech + text model on an RTX 5090. Stage A fine-tunes the top 4 layers of WavLM-base-plus and the top 2 of DeBERTa-v3-base, with acoustic scalars and modality dropout, on emotion, arousal and delivery. 55,150 segments from 2,251 calls across AMI, MELD, RAVDESS, LibriSpeech, Diplomacy and synthetic ElevenLabs calls, each dataset weighted equally per epoch. Separate learning rates for encoders and heads (2.8e-5 / 1.4e-3), 5% warm-up then cosine decay, bf16 autocast, fused AdamW. On held-out groups, fusing audio and text beat either alone: emotion 0.623 accuracy (text only 0.547, audio only 0.482, majority 0.437); arousal 0.798 (majority 0.587). Found why one head collapsed: a class weight on a class with no training labels, combined with label smoothing, pushed it to always predict that class.
- Model training: A deepfake classifier that can't cheat on identity. Frozen WavLM embeddings, mean-pooled over 3 s windows (1.5 s hop), into a standardised logistic regression. A whole speaker is held out for testing, so the model has to learn real vs fake rather than who is speaking. Telephone augmentation: an 8 kHz round trip band-limits training audio like a phone line. Reports AUC and equal error rate per window and per file.
- Model architecture: Signals first, readings second. Per-turn signals (pitch, loudness, pace, pauses, fillers) and their change against the speaker's own earlier turns feed Stage B, a causal GRU over the conversation. It only sees the past, so it can run live, and it adds optional readings that report insufficient evidence when they have too few labels.
- Systems: The live pipeline. Silero voice activity → local Whisper transcript, ECAPA voice match, deepfake score and delivery against the speaker's own baseline, plus LLM and rule-based fraud patterns, fused into one trust score with live warnings and a signed report.
- Python: Detectors, fusion, the training notebooks and scripts, FastAPI back end and evaluation.
- TypeScript: The React call UI and report pages.
- Code: https://github.com/ANonABento/baseline

### GymBuddy (Oct 2026)

An AI-native iOS gym and nutrition coach: a deterministic, unit-tested Swift engine plus Claude meal parsing from text, voice and photos.

- Context: Own project
- Evidence: 73 GymBuddyCore unit tests pass (swift test) and 15 end-to-end UI flow tests pass in the simulator (Oct 2026).
- Areas: iOS, Applied AI
- Languages: Swift, TypeScript
- Stack: Swift, SwiftUI, TypeScript, Cloudflare Workers, Claude
- Model training: Learned food pairings. A food2vec model trained offline over recipes scores which ingredients go together; the on-phone generator fills protein, carb, vegetable and flavour slots by pairing score minus a nutrition penalty, then a linear program solves the grams within each ingredient's portion bounds.
- Systems: A deterministic engine under the AI. The coaching maths (progression, e1RM trends, trend weight, TDEE, macro targets, program planning) is a pure Swift package with no UI or AI code, so it is unit-tested on its own. Claude parses meals from text, voice and photos and writes meal plans, but the back end sums each day's macros itself and asks for one revision if a day is more than 5% off on calories or 10 g off on protein.
- Swift: SwiftUI app, the coaching engine package, widgets, a rest-timer Live Activity and Apple Health sync.
- TypeScript: Cloudflare Worker: food search, barcode lookup and the Claude endpoints.
- Python: Offline training of the food-pairing model.
- Code: private repository, available on request

## Signals by area

- Computer vision: RocScience (Computer Vision Engineer), WALDO, Core Segmentation, Trash Can Robot, Distance Scorer
- Robotics: RLWRLD (Forward Deployed Engineer), WALDO, WATO ASD Robot, Trash Can Robot
- Machine learning: RocScience (Computer Vision Engineer), Core Segmentation, Distance Scorer, Baseline
- Model training: RLWRLD (Forward Deployed Engineer), RocScience (Computer Vision Engineer), Distance Scorer
- Security: White Hat Consulting (Cybersecurity Engineer), WreckLess, Baseline
- Applied AI: WreckLess, The Flight of the Buffalo, GymBuddy
- Embedded: WALDO, Trash Can Robot
- Simulation: WATO ASD Robot, The Flight of the Buffalo
- Teleoperation: RLWRLD (Forward Deployed Engineer)
- Robot learning: RLWRLD (Forward Deployed Engineer)
- Data engineering: White Hat Consulting (Cybersecurity Engineer)
- CAD: WALDO
- AI agents: WreckLess
- Path planning: WATO ASD Robot
- Lab automation: The Flight of the Buffalo
- Audio AI: Baseline
- iOS: GymBuddy