H2-1 — System Definition, Modalities, and Engineering Boundary

Engineering boundary (what this page covers)

An interactive wall/whiteboard is defined as a coordinate generation system that converts real-world actions (finger/pen/object) into timestamped 2D coordinates with confidence, event type, and (when needed) pen/finger identity. The hardware boundary is the chain from sensor excitation & AFE through sampling/timestamps to stable coordinates delivered to the host interface.

Interaction requirements expressed as measurable outputs

Sensing modality map (not a feature list)

Each modality is evaluated using the same engineering lenses: capability (touch-only vs pen+hover), environment stress (sunlight/reflective surfaces/LED flicker), occlusion sensitivity, geometry error sources (parallax & mounting tolerance), and factory calibration complexity. This prevents the content from becoming a generic “technology catalog”.

Note: “Best for” assumes the synchronization backbone is solid. If timestamps/frame boundaries are inconsistent, hybrid systems can look worse than single-modality systems due to fusion conflicts.

Must-hit KPIs: define, budget, and verify

Latency must be treated as a budget, not a single number. Interaction “feel” is dominated by p95 jitter, not average FPS. Accuracy must be split into center error and corner error, because corners amplify parallax and calibration residuals. False touches must be expressed in a repeatable metric (events/min under defined light/noise conditions) to avoid subjective debates.

Output: Choose-your-modality decision box (engineering switches)

H2-2 — Reference Architecture and Signal-Chain Map (End-to-End)

Why this chapter exists (avoid “fragmented” thinking)

Interactive walls are multi-sensor systems. Failures such as corner misalignment, coordinate swaps, and latency spikes often originate from timing inconsistencies rather than sensor sensitivity alone. A canonical reference architecture prevents later sections from turning into isolated, hard-to-integrate blocks.

Two parallel backbones: Signal Chain vs Synchronization Chain

Sensor-to-host architecture blocks (what each block must guarantee)

Each sensing modality generates events in a different sampling domain: ToF produces time/phase-derived distance features, cameras produce frame-based observations, and touch/pen controllers produce scan-based coordinates. The architecture must guarantee:

• Deterministic timebase: a common counter or synchronized timestamp source for all event producers.
• Frame boundary integrity: stable cadence (no silent drops) for camera/touch sampling.
• Alignment policy: a defined rule for fusing asynchronous sources (e.g., align all sources to camera frame time, then interpolate touch).
• Backpressure visibility: measurable indicators for queue growth, DMA contention, and dropped frames.

Output: “What must be synchronized” checklist (mechanical and testable)

Interfaces: mention only what affects latency, sync, and integrity

High-speed image sensors typically connect through CSI-class interfaces; touch/pen controllers often use I²C/SPI; triggers and sync use GPIO/FSYNC lines. The engineering focus is not protocol depth, but: bandwidth headroom, latency determinism, timestamp availability, and noise/ESD resilience on long harnesses and metal frames.

Evidence-first instrumentation points (to avoid blind debugging)

These counters/logs should be available in factory validation and field debug. If they do not exist, intermittent failures tend to be misdiagnosed as “sensor quality issues”.

H2-3 — Laser/ToF Positioning AFE: Tx/Rx, Timing, and Ambient Light Rejection

Why ToF “works in lab” but fails in bright classrooms

Field failures usually trace back to margin collapse in the optical front-end: ambient light increases shot noise and pushes the receiver toward saturation; reflections introduce multipath; contamination reduces optical throughput; and poorly defined gating/timestamps turn small analog issues into large coordinate jitter.

Tx chain: pulse energy, edge quality, and repeatability

The transmitter determines how many photons return to the receiver. The engineering objective is repeatable optical pulses with controlled peak current and clean edges. Pulse edge uncertainty converts directly to time jitter, while protection/derating events convert to energy variability and unstable histograms.

Rx chain: dynamic range + saturation recovery define real-world robustness

The receiver must handle a large spread of returned signal levels while ambient light elevates the background. Dynamic range must be evaluated together with saturation recovery time. A receiver that clips and recovers slowly will create “phantom peaks” and time shifts in subsequent measurements.

Timing: TDC/ADC sampling, histogram shaping, gating windows, multipath signatures

Time-of-flight estimation should be treated as a shape measurement: peak position, width, and floor matter. Gating is the main tool to reject unwanted paths—done incorrectly, it can lock onto the wrong reflection and make calibration appear “broken”.

• Histogram ToF: interpret peak position (range), peak width (jitter/noise/multipath), and floor (ambient shot noise).
• Gating windows: early windows risk direct stray reflections; late windows risk multipath; window policies must match geometry.
• Multipath: watch for double peaks or peak shifts that correlate with angle, glossy surfaces, or occlusions.

Ambient rejection: optical filtering + modulation + flicker immunity

Ambient light increases the histogram floor and can force the AFE into clipping. The practical countermeasures are optical filtering, modulation, and flicker-aware sampling. Flicker interference often presents as periodic drift or periodic dropouts under LED/fluorescent lighting.

Output A — ToF SNR margin worksheet (what eats the margin)

Output B — First three waveforms to capture (minimum evidence set)

H2-4 — Camera ISP Path for Interaction: Sensor Choice, Sync, and Motion/Lighting Failure Modes

Camera is not “add a camera”: shutter, exposure, and ISP stability dominate interaction feel

Camera-based interaction quality is dominated by temporal correctness: shutter type, exposure time, timestamp definition, and ISP pipeline choices determine whether fast strokes remain stable and whether latency stays predictable under changing lighting and occlusion.

Sensor choice: global vs rolling shutter (expressed as interaction consequences)

ISP blocks: what helps vs hurts interaction (focus on determinism, not beauty)

Interaction pipelines prefer stable, low-latency processing over aesthetically pleasing images. Heavy temporal filtering can introduce lag/ghosting. Aggressive noise reduction can erase pen-tip features. Unstable AE/AWB can shift thresholds and cause intermittent dropouts or false detections.

Frame sync: trigger + timestamp definition + illumination coupling

Frame sync is the root cause layer. A robust design defines where timestamps are taken (exposure start or frame arrival), enforces consistent frame boundaries, and prevents IR strobe/illumination from creating flicker banding or periodic coordinate drift.

Failure modes: symptom → evidence → likely chain

Output A — ISP “safe-mode” list for interaction (minimum latency + stable exposure)

Output B — Evidence checklist (minimum set for root-cause isolation)

If these are not logged/observable, camera issues are frequently misattributed to “insufficient compute” rather than shutter/sync/ISP instability.

H2-5 — Touch Sensing on Large Surfaces: IR Grid / Optical / Capacitive (Board-Side)

Engineering boundary: touch accuracy is geometry + noise immunity + controller stability

Large-surface touch systems fail in the field mainly due to noise injection and geometry instability, not because “touch theory” is missing. Robust designs treat touch as a signal chain: sensing physics → interference paths → controller scan/baseline behavior → coordinate output.

IR grid touch: spacing, alignment drift, sunlight blind spots, bezel reflections

IR grid touch is geometry-driven: emitter/receiver spacing and mechanical alignment define the nominal resolution. Field robustness depends on maintaining alignment and protecting receiver dynamic range under sunlight and reflective bezels.

Optical touch: finger detection confusion cases (shadows, sleeves, specular highlights)

Optical touch failures are typically classification failures driven by lighting and occlusion: shadows resemble “touch blobs”, sleeves create large occlusions, and specular highlights shift thresholds. Stable capture cadence and conservative segmentation policies outperform “pretty images” for interaction determinism.

Capacitive large panel: common-mode noise, display interference, water/ESD robustness

Capacitive touch on large panels is dominated by common-mode noise and coupling from nearby high-energy systems. Display switching noise, PSU ripple, and long harness coupling can destabilize the baseline and trigger false touches. Water films alter the electric field distribution and stress baseline tracking; ESD events stress recovery and filtering.

Controller selection: scan rate, multi-touch count, latency, and baseline tracking

Controller selection must be framed as a KPI decision. The key is not “how many channels exist”, but whether the controller can sustain required scan cadence, multi-touch count, and baseline stability under interference.

Output A — Touch error budget (parallax + baseline drift + noise-induced false triggers)

Output B — “EMI suspects” list (display noise, PSU ripple, harness coupling)

H2-6 — Touch/Pen Fusion and Coordinate Mapping (Homography, Parallax, Drift)

Why corner offset grows: maximum extrapolation + parallax + drift accumulation

The typical field complaint is “center is fine, corners drift out.” This happens because corners represent the maximum projection angle and maximum model extrapolation, where small errors in geometry, timing, and baseline become large coordinate bias. Mapping quality is therefore defined by the full transform stack, not any single sensor.

Coordinate transforms: camera / ToF / touch → wall plane

Each modality produces coordinates in its own measurement plane. Robust systems explicitly define transforms into a single wall-plane coordinate and track where errors enter: lens distortion and mount flex (camera), multipath and gating mistakes (ToF), and baseline/CM noise (touch).

Homography calibration (practical): points, placement, corner weighting

Homography calibration quality depends mostly on where calibration points are placed. Center-only calibration underfits edge geometry. A practical approach is to cover center, mid-edges, and corners so the model does not rely on unstable extrapolation. Corner weighting improves user-perceived quality because corner errors dominate “writing feels wrong” complaints.

Parallax: sensor depth vs surface creates location-dependent error

Parallax is a geometric error caused by the sensor being offset in depth from the interaction surface. The same physical touch can map differently depending on position, with corner regions most sensitive to depth mismatch. Small mechanical changes (wall flatness, frame flex, adhesive creep) change the parallax model and create drift.

Drift sources: temperature, mechanical flex, lens shift, adhesive creep

Drift is best treated as a time-axis error budget. Warm-up phases can show fast drift, while steady-state drift is slower. Validation should track max drift per hour and define re-calibration triggers after transport, mounting changes, or ESD events.

Output A — Step-by-step calibration procedure + acceptance thresholds

Output B — Corner error triage flow (geometry vs timing vs exposure vs baseline)

H2-7 — Pen Inputs on Interactive Boards (Board-Side Digitizer & Hover)

Pen technologies commonly seen on walls (board-side view)

Interactive boards encounter multiple pen modalities. The key engineering lens is what the board can measure reliably: coordinate, confidence, and timing. Each modality has distinct hover SNR sensitivity, occlusion behavior, and collision failure modes.

Hover detection: why it breaks first

Hover is the first feature to fail because it runs at the weakest signal condition: small target, weaker coupling/reflectivity, and a detection threshold close to the noise floor. Ambient changes and occlusion reduce the signal margin rapidly, causing jitter and dropouts before contact tracking fails.

Pen ID / multi-pen: tagging and collision cases

Multi-pen behavior fails when signals overlap in space or time. The most frequent issues are ID collisions (same marker pattern/frequency), track swaps when two pens cross, and selective hover loss where the weaker pen drops first.

Output A — Pen tracking stability checklist (hover SNR, occlusion, high-speed stroke)

Practical test patterns: (1) stationary hover at multiple distances/angles, (2) fast straight strokes and sharp turns, (3) two-pen crossing paths with occlusion events, with confidence/ID and dropout counters logged.

Output B — Pen vs finger conflict arbitration rules (HW/FW boundary)

Arbitration belongs at the boundary where board-side digitizer signals meet touch events: it defines which modality owns the coordinate stream and prevents “double input” (a pen stroke generating both pen and finger touches). A robust policy prioritizes the highest-confidence modality and uses hold/release hysteresis to avoid flapping.

H2-8 — Processing SoC Selection: Latency Budget, Hardware Acceleration, and I/O Topology

SoC selection is a latency/jitter problem, not “pick a fast chip”

Interaction feel is governed by median latency and p95 jitter. A SoC that looks fast on average can still feel laggy if memory bandwidth is saturated, DMA queues are congested, or sensor synchronization is weak. A correct selection starts from the end-to-end pipeline and budgets milliseconds per stage.

Pipeline stages: capture → ISP → detection → fusion → output

Key hardware blocks to check (must-have vs acceleration)

Latency and jitter: where worst-case spikes come from (hardware-facing)

Worst-case latency spikes typically come from contention: multiple high-rate sensors writing to DDR while ISP and detection read/modify buffers, leading to queue growth. DMA priority limits and insufficient bandwidth headroom turn temporary bursts into visible interaction lag.

Output A — Latency budget table (targets: median + p95)

Replace “low/moderate/bounded” with product-tier numbers later; keeping both median and p95 prevents “fast average but laggy feel” outcomes.

Output B — Bandwidth sanity check (camera + ToF + touch concurrency)

I/O topology: CSI lanes, sync GPIO, and touch controller buses

Interaction systems require concurrency and synchronization. SoC I/O selection should verify: enough CSI lanes for simultaneous sensors, stable sync GPIO for triggers/timestamps, and robust buses for touch/digitizer controllers under long-cable noise.

H2-9 — Power, Grounding, EMC/ESD for Large-Format Interaction Modules

Bench pass vs wall-mount failure: what changes physically

A large metal frame and long harnesses introduce new return paths and coupling capacitances. The same electronics can shift from stable to fragile when: (1) chassis/frame becomes a strong reference and ESD return route, (2) cable shields create common-mode currents and ground loops, and (3) display switching noise and power transients couple into weak-signal sensing rails.

Power rails: keep weak-signal sensing separated from bursty digital loads

Interaction modules typically mix weak analog sensing (ToF/optical receivers, touch/EMR front-ends) with high-burst compute (SoC/DDR/ISP). Stability depends on rail partitioning and the ability to prevent digital current steps from moving the sensing reference.

Sequencing and inrush are only “brief” topics here, but the practical impact is clear: a marginal rail during bring-up can lock the system into a bad sensing baseline, making later calibration appear inconsistent.

Grounding: frame ground loops, shield termination, and sensor reference

Large-format installations add a chassis reference (metal frame) that can dominate return currents. The main goal is to control where common-mode currents flow and prevent shield and chassis returns from crossing sensitive sensing references.

EMC: display switching noise + long harness = coupling amplifier

The most frequent interaction-chain EMC issue is not external RF, but internal coupling: display switching noise and power ripple find paths into AFEs and sensor references. Long harnesses act as antennas and common-mode conduits, turning small disturbances into repeatable tracking instability.

ESD and protection: distinguish latch-up vs saturation vs reset

ESD to bezels and sensor windows is common in classrooms. The failure signature depends on the return path and the victim node. Protection components (ESD diodes/TVS) must be selected not only for clamping, but also for their side-effects on high-speed and weak-signal lines.

Output A — Noise isolation checklist (power filters, ground partition, cable strategy)

Output B — ESD evidence capture (logs/waveforms that separate root causes)

H2-10 — Validation & Production Test Plan (Accuracy, Latency, Sunlight, Multi-User)

Turn requirements into repeatable pass/fail

A useful validation plan defines (1) test conditions that can be reproduced, (2) measurable metrics that map to user feel, and (3) pass criteria that catch p95/p99 failures early. The same structure supports production end-of-line tests and drift monitoring.

Accuracy: grid mapping + corner weighting + dynamic stroke

Accuracy must be evaluated both statically and dynamically. Large surfaces often fail first at corners due to geometry and reference drift, so corners require higher test density and stricter acceptance checks.

Latency: measure both median and p95 jitter

Median latency defines baseline responsiveness, while p95 jitter defines perceived “stutters.” Two conceptual approaches are common: high-speed camera (event-to-pixel) and timestamp loopback (event-to-output cadence). The measurement method matters less than a consistent jitter report.

Sunlight/lighting: lux levels, flicker, reflections, IR contamination

Lighting tests must include lux intensity and temporal artifacts. Sunlight and flicker raise the detection noise floor and can distort tracking confidence, while window reflections create structured false features. IR contamination is especially relevant for IR grid, IR markers, and ToF-assisted paths.

Reliability: thermal drift, vibration/impact, contamination

Reliability tests focus on drift and repeatability. Thermal drift changes alignment and baselines; impacts can shift geometry or loosen shielding paths; contamination reduces optical contrast and collapses hover margin first. Each condition must be paired with measurable drift metrics.

Production: factory calibration flow, EOL tests, self-test hooks

Output A — Test matrix table (condition × metric × pass criteria × instrumentation)

Replace template pass criteria with product-tier thresholds; keep the same matrix structure across validation, regression, and production.

Output B — “Golden unit” strategy and drift monitoring