H2-1. What This Controller Does and Where It Fits

A Batch / Recipe Controller is the execution core that turns a versioned recipe into a repeatable run: it orchestrates step timing, I/O sequencing, metering windows, time-stamped events, and evidence-grade batch records, then integrates the results to SCADA/HMI/historian/MES via Ethernet or fieldbus—without replacing PLC safety logic.

To avoid scope confusion, treat industrial automation as four layers with distinct responsibilities:

• Field layer (sensors, valves, actuators, meters): produces raw signals and executes physical actions.
• Control layer (PLC, remote I/O, safety interlocks): guarantees basic deterministic I/O control and failsafe defaults.
• Orchestration layer (Batch/Recipe Controller): defines and proves the execution semantics of a batch run—steps, phases, timeouts, retries, checkpoints, resumes, and operator interventions.
• Supervisory/enterprise layer (HMI/SCADA/historian/MES): visualizes, aggregates, and connects business context; it should not be the source of truth for step-by-step execution evidence.

The practical boundary is evidence and repeatability. If you need to answer “what exactly happened” at each step (with timestamps, recipe version, operator actions, and measured totals), you need a controller that owns: recipe → batch instance → step actions → event log.

Use an independent Batch/Recipe Controller when:

• Recipes change and must be traceable (versioning, approval, rollback, effective range per batch).
• Steps are complex (branching, parallel phases, resource locks, retries, conditional waits).
• Metering is contractual (totalizers, tolerances, reconciliation across devices/data sinks).
• Batch records are audited (operator actions, parameter changes, exception handling, power-loss continuity).
• Equipment spans buses (Ethernet + fieldbus devices with unified sequencing and health monitoring).

A PLC alone is typically sufficient when:

• Logic is stable, steps are few, and there is no need for recipe version governance.
• Metering is for display only (no reconciliation loop, no audit requirement).
• Power-loss recovery can restart the process without step-level resume requirements.

Design anchor: separate “hard safety” from “batch orchestration.” Keep safety interlocks and fail-safe outputs in the PLC/safety chain; let the Batch/Recipe Controller focus on deterministic sequencing and evidence-grade records.

• batch_id
• recipe_version
• event_ts (monotonic + UTC)
• meter_window_id
• step_state (start/end/abort/resume)

H2-2. Scope Guard and Acceptance Criteria

This section converts “batch control” from a concept into sign-off-able engineering criteria. Each line below is written as: metric (what to measure) + evidence (what to capture) + first fix (what to change first).

Acceptance checklist (10 lines):

• Step timing accuracy — specify max jitter and timestamp resolution; capture step_start/step_end deltas; first fix: prioritize sequencer + async logging.
• Event continuity under power loss — define max loss window (e.g., ≤N seconds); capture power-fail test log + recovery proof; first fix: journal + checkpoint cadence.
• Recipe version governance — define versioning, approval, rollback; capture who/when/why + effective batch range; first fix: bind recipe_version immutably per batch_id.
• Metering reconciliation — define allowable batch total error; capture totalizer vs meter vs historian comparison; first fix: lock sampling window + calibration version per batch.
• Comms resilience — define reconnect time + stale-data marking; capture drop/reconnect replay; first fix: watchdog + offline queue + explicit stale state.
• Interlocks and safe stop — define permissives, timeouts, safe outputs; capture interlock test cases and safe-state table; first fix: keep hard safety in PLC/safety chain.
• Audit trail completeness — require who/what/when/why/impact; capture export sample linked to batch_id; first fix: route all changes through a single audited API.
• Storage lifetime — estimate years at events/sec and write amplification; capture wear model + leveling strategy; first fix: event tiering + batch commits + verbosity caps.
• Time-base boundary — define RTC/NTP/PTP roles + sync health; capture offset/holdover/sync_state in logs; first fix: dual timestamps (monotonic + UTC) per event.
• Commissioning readiness — require simulator/replay/self-test; capture dry-run report + deterministic replay; first fix: build test-first with recorded trace playback.

Scope guard: this controller must not become “everything.” It owns sequencing + evidence, while PLC owns failsafe I/O, and SCADA/MES own visualization and business workflow. That separation is itself an acceptance criterion because it prevents untestable coupling.

H2-3. Data Model: Recipe → Batch → Step → Action

A robust batch system is built on an executable, auditable data model. The core rule is to separate the static contract (Recipe) from the runtime instance (Batch), then break execution into provable units (Step/Phase) and replayable atoms (Action).

Recipe (static template) defines the approved structure and parameters. It must be versioned and immutable per approval cycle. A recipe is not “a spreadsheet of fields”; it is a graph of steps with branching/parallelism rules, entry/exit conditions, timeout policies, and parameter constraints.

Batch (runtime instance) binds to one recipe version and records what happened in the real world. This binding is what makes audit and troubleshooting possible: a batch_id must always map to a single recipe_version.

Step / Phase (execution unit) is the smallest unit that should have a clear start/end boundary, a measurable outcome, and a deterministic transition rule. Steps can be sequential, parallel, looped, or conditional—yet still must be explainable via logged state transitions.

Action (replayable atom) is the minimum work item the sequencer executes: setting I/O states, applying setpoints, opening a metering window, verifying stability, waiting for a condition, or emitting a log marker. Actions must carry enough parameters to be replayed in simulation or trace playback.

• recipe_id
• recipe_version
• batch_id
• step_id
• action_id
• meter_window_id
• event_seq
• event_ts_mono
• event_ts_utc
• calib_version

H2-4. Sequencing Engine: Determinism, Concurrency, and Interlocks

Determinism is not a claim—it is a design property that can be measured. A sequencing engine is deterministic when the same inputs and time base produce the same step transitions, and when every transition can be explained by logged evidence (events, interlocks, timeouts, and resource arbitration).

Scan loop vs event loop: treat them as a boundary decision, not a debate. Periodic sampling is useful for filtering and stable condition checks, while event-driven dispatch is essential for bus updates, metering window edges, timeouts, alarms, and operator interventions. A practical architecture uses a single prioritized event queue where both periodic ticks and asynchronous signals land.

Concurrency is controlled by resources, not hopes. Parallel steps must declare what they own: exclusive actuators (valves/pumps), shared sensors, and rate-limited services (storage/network). The sequencer must provide resource locking and arbitration so conflicts produce predictable outcomes.

Interlocks/permissives must be evidence-grade. A raw input should not flip a batch state instantly. Use a condition tree with explicit debounce windows, optional voting windows, and a defined trip latch policy (latched until operator reset vs auto-clear).

Timeouts define safety and recoverability. Separate: soft abort (stop progression and preserve diagnostics), hard abort (force safe outputs, typically via PLC/safety chain), and the safe state definition (what every output becomes). Recovery is then a choice: resume from a step checkpoint or restart the batch, based on checkpoint granularity and metering state.

• event_queue_depth
• dispatch_latency_ms
• resource_id
• owner_step_id
• wait_reason
• interlock_id
• debounce_ms
• vote_window_ms
• timeout_ms
• retry_count
• checkpoint_id

H2-5. I/O Sequencing: Commissioning-Friendly Mapping & Safety Defaults

Commissioning becomes predictable when I/O is not hard-coded. A maintainable architecture separates logical intent (process meaning) from physical realization (wiring / addresses), with a versioned mapping layer that is auditable, testable, and safe by default.

I/O mapping layers: define process-facing tags such as Valve_01_Open or Pump_A_Start, then bind them to physical channels (local I/O, remote I/O, or fieldbus registers) via a single mapping contract. This contract should include polarity, scaling, and protocol binding, plus lifecycle controls (mapping_version, approvals, and effective dates).

Safety defaults (fail-safe outputs) must be explicit per output and per fault category. “Safe” is not one value—it is a policy: drop vs hold, latched vs auto-clear, and the recovery rule. Hard safety trips should be enforced by PLC/safety chains; the batch controller should orchestrate orderly stops and log the evidence of every transition.

Input semantics determine deterministic behavior: edge vs level, debounce windows, and filter principles must be part of the design. Raw inputs should not drive step transitions directly; they should pass through debounce/filter stages and emit events that can be traced in logs.

Manual override is a controlled mode, not ad-hoc output toggling. Enter/exit conditions, role-based permission, and time-bounded overrides are required. Every manual action must create an audit event that links back to batch_id/step_id where applicable.

Dry-run and forced I/O enable safe commissioning and reproducible troubleshooting. Dry-run advances the sequencer without driving physical outputs. Forced I/O must be time-limited, clearly flagged, and recorded with who/when/why, so batch evidence remains trustworthy.

• logical_tag
• protocol_binding
• physical_channel
• polarity
• scaling
• mapping_version
• safe_value
• recover_policy
• forced_flag
• override_role

H2-6. Metering & Timing: Sampling Windows, Totalizers, and Reconciliation

Metering is the proof layer of a batch. The goal is not “a number on a screen” but a result that can be recomputed, reconciled, and audited. That requires explicit sampling windows, stable time bases, versioned calibration, and reconciliation outputs.

Instantaneous vs totalized: instantaneous measurements are noisy and time-dependent; totalized values depend on the integration method and the accuracy of timestamps. Drift typically comes from sampling cadence mismatch, filter delay, time-base jumps, or calibration mismatch—so all of those must be visible in evidence logs.

Sampling windows define where metering starts and stops. Windows should be triggered by step events and captured with monotonic timestamps for ordering, plus UTC timestamps for correlation across systems. Filtering should be minimal and traceable (type + parameter), so totals can be recomputed in offline replay.

Metering events should be explicit: start/stop, threshold reached, and stability conditions. For example, a stable condition can be defined as “variance below a tolerance for a continuous duration,” which prevents transient spikes from closing a window early.

Reconciliation compares three totals: (1) batch total computed from the window, (2) meter-device totalizer, and (3) historian total. The output should include PASS/WARN/FAIL plus a delta value and a hint category (window mismatch, calibration mismatch, missing samples, time jump).

Calibration must be versioned. Every metering window must record the calibration version (and validity), otherwise the batch record cannot prove what coefficients were used when the result was produced.

• meter_window_id
• sample_period_ms
• filter_type
• window_start_event
• window_stop_event
• totalized_value
• calib_version
• meter_total
• historian_total
• reconcile_status

H2-7. Time Base: RTC, NTP, PTP/1588 and Timestamp Strategy

A reliable batch record needs two different “time truths”: ordering time and correlation time. Event ordering must be based on a monotonic clock to survive NTP/PTP steps, while cross-system correlation uses UTC wall time plus a recorded sync state.

RTC provides a retained time baseline across power loss. It is not a precision sync tool, but it prevents wall time from resetting to an invalid value after reboot. NTP is appropriate for general synchronization where millisecond-level alignment is acceptable, but it can introduce time steps (sudden jumps). PTP/1588 targets sub-millisecond alignment and is preferred when multiple devices must agree on window edges, sequence timing, or fine-grained measurements.

Timestamp rule: every evidence-grade event should record a dual timestamp: ts_mono_ns (monotonic for strict ordering) and ts_utc (UTC for correlation), plus sync_state and offset so time quality is visible during audits. When the system enters holdover or becomes unsynchronized, the record must still be continuous and clearly marked.

Clock health should be observable and actionable: capture sync_state, offset, last_sync_ts, holdover_age, and time_step_count. If sync quality degrades, the controller can apply a deterministic downgrade policy (e.g., more conservative metering decisions, reconciliation marked WARN, or operator confirmation for critical changes).

Multi-source redundancy avoids single points of failure. A practical priority policy is: PTP_LOCKED > NTP_SYNC > HOLDOVER > RTC_ONLY. Any source switch should generate a dedicated event so later investigations can explain time-quality changes.

• ts_mono_ns
• ts_utc
• sync_state
• offset_ms
• holdover_age_s
• last_sync_ts
• time_step_count
• time_source
• primary/backup

H2-8. Event Log & Audit Trail: Evidence-Grade Records (EBR-ready)

Evidence-grade logging is not “debug text.” It is a structured chain that can answer: who did what, when, with which version, and which batch was affected. Logs should be linkable (join keys), exportable, and tamper-evident.

Event classes should be audit-oriented: process events (step transitions, metering window edges, interlock trips), configuration changes (recipe releases, mapping_version changes, thresholds), and security-relevant actions (manual override, forced I/O, role changes, auth failures). Each class requires different mandatory fields, but all must include batch_id and trustworthy timestamps.

Tamper-evidence can be achieved with a lightweight hash chain: each event stores hash_prev and a computed hash_curr over key fields. Optional signatures can be applied to high-value milestones (e.g., batch completion or recipe publish) without signing every log line.

Capacity strategy should be tiered and deterministic: a RAM ring buffer for high-frequency debug traces, an append-only flash log for durable evidence events, and a server/historian tier for long-term retention and search. If a loss window occurs (power failure before flush), the system should record the loss interval explicitly.

Join keys are the backbone of traceability: batch_id, recipe_version, operator_id, device_id, plus event_seq and step_id/action_id to localize cause. Mapping_version and calib_version should also be captured, so I/O behavior and metering results remain provable in audits.

Export should be simple and consistent: CSV/JSON for evidence packages, and structured feeds to SCADA/historians (tags or topics) for monitoring. Export should never strip the fields that prove integrity and linkage.

• event_seq
• event_type
• hash_prev
• hash_curr
• batch_id
• recipe_version
• operator_id
• device_id
• mapping_version
• calib_version
• export_ref

H2-9. Recipe & Event Storage: Media Choice, Wear, Power-Fail, and Integrity

Storage in a batch controller is not “saving files.” It is an engineering mechanism that must support high-frequency writes, long service life, power-fail consistency, and verifiable integrity. The design starts by separating workload types: recipes are versioned and read-mostly; events are append-heavy and must remain replayable after a sudden reboot.

Workload split: a recipe is a compact, versioned artifact (approve, publish, rollback), so it should be stored in a slot-based or image-based format that supports atomic activation (A/B slot or dual-image). An event log is a continuous stream; it should be written as an append-only journal with explicit commit markers so incomplete tails can be detected and truncated after power loss.

Power-fail consistency requires a deterministic commit path. For event journal entries, a safe pattern is: write payload → write length/CRC (or hash) → write commit marker → advance pointer atomically. For recipes, write the new version into an inactive slot, validate it, then flip an active pointer as the final step. Recovery should replay from the last checkpoint plus verified journal entries.

Wear and lifetime are controlled by minimizing write amplification and avoiding hot spots. Depending on the write rate and retention needs, media choices may include FRAM/MRAM for high-endurance logging, or NOR/SD for larger capacity when paired with journal-structured writes and explicit health monitoring. When media is block-based, bad-block handling and a clear allocation strategy are required.

Integrity and rollback must be evidence-ready. Every stored recipe should include version metadata, and every event should carry a checksum and sequence number. If a rollback occurs, it should create a dedicated audit event with from/to versions and a reason, so downstream reports can explain changes in batch outcomes.

Tiering improves resilience: keep hot, queryable state locally (recent events + current batch snapshot), store durable evidence in append-only flash media, and export cold archives to a server/historian. Offline buffering should be treated as a normal mode, with explicit “catch-up” export after reconnection.

• recipe_version
• active_slot
• event_seq
• commit_marker
• crc/hash
• checkpoint
• recovery_event
• write_amplification
• wear_index
• bad_block_count
• storage_health_state

H2-10. Ethernet & Fieldbus Integration: Mapping, Latency, and Failover

Connectivity is not “link up.” Integration must define a tag contract, a connection state machine, a latency budget, and a failover behavior. Control decisions must remain deterministic even when data becomes stale or links flap.

Protocol stacks may include Modbus TCP, EtherNet/IP, PROFINET, or OPC UA, but the engineering value is above the protocol: a versioned tag map with stable naming, units, scaling, and compatibility rules. The controller should expose tag_map_version and device profiles so commissioning changes can be audited and rolled out safely.

Connection management must be explicit: reconnect loops, watchdog timers, and stale-data marking. “Stale” is not just missing updates; it means the value must not be used for step gating or safety decisions. Each tag should have an age_ms (or last_update_ts_mono) and a quality flag (GOOD/STALE/BAD) so behavior is explainable in logs.

Latency requires plane separation: keep control-plane updates isolated from telemetry-plane reporting. Control-plane traffic is bounded and deterministic (step gating, key setpoints), while telemetry can be buffered and bursty (trends, historian feeds). Telemetry congestion must not affect control decisions.

Failover should preserve safety and evidence: link switches, dual networks, or redundancy principles (e.g., parallel paths) must trigger a link_failover_event, and quality flags should show degraded states during switching. After reconnection, the system should emit re-sync completion events and reconcile any buffered telemetry exports.

• tag_map_version
• device_profile_id
• quality
• age_ms
• watchdog_state
• reconnect_count
• stale_flag
• control-plane
• telemetry-plane
• link_failover_event
• re-sync

H2-11. Validation & Commissioning Playbook (Field-Ready Tests)

This playbook turns system claims into repeatable evidence. Each test is written as a 3-line SOP: Goal (what must be proven), Method (how to reproduce), and Evidence (which fields/waveforms/exports to capture). Pass criteria are explicit so commissioning results are auditable.

Example MPNs for commissioning setups (reference BOM snippets): The parts below are commonly used building blocks to implement or validate the mechanisms above (timebase, storage, power-fail, network I/O, tamper-evident logs). Select equivalents per platform and availability.