music-collaboration-and-networking
Designing Redundant and Resilient Aes67 Networks for Critical Broadcast Applications
Table of Contents
In mission-critical broadcast environments—live news, emergency alert systems, and major sports events—any interruption in audio can be catastrophic. The shift from traditional analog and MADI infrastructure to Audio-over-IP (AoIP) demands networks that are not only high-performance but also engineered for fault tolerance. AES67, the interoperability standard for high-performance audio-over-IP, provides the foundation for these systems. However, deploying AES67 in critical applications requires deliberate design around redundancy and resilience to ensure that hardware failures, link outages, or network congestion go unnoticed by listeners.
This article expands on the core principles of building a redundant and resilient AES67 network, covering path redundancy, device failover, timing protection, and practical implementation steps. By combining protocol-level mechanisms like SMPTE ST 2022-7 with robust physical topologies and proactive monitoring, broadcast engineers can achieve the five-nines reliability that modern broadcasting demands.
Understanding AES67 and Its Role in Critical Broadcast
AES67, published by the Audio Engineering Society, defines an interoperability framework for transmitting high-quality audio over IP networks using RTP. It standardizes sample rates (48 kHz), bit depths (16 or 24), and packet times (125 µs to 1 ms), enabling devices from different manufacturers to exchange uncompressed or lightly compressed streams. In practice, AES67 is the underlying transport for many AoIP ecosystems, including Livewire, Q-LAN, and Ravenna, and is often used alongside SMPTE ST 2110-30 for professional media networks.
For critical broadcast applications, the stakes are high. An audio dropout during a live election coverage or a Super Bowl broadcast can lead to revenue loss, reputational damage, and regulatory scrutiny. Therefore, any AES67 deployment must start with a clear understanding of failure points: physical layer (cables, connectors), network switches, power supplies, audio devices, and timing sources. Each layer requires specific redundancy techniques to maintain continuous operation.
Key Principles for Designing Redundant AES67 Networks
A truly redundant AES67 network eliminates single points of failure at every level. The following subsections break down the most critical design principles.
Network Path Redundancy
Single-path topologies are unacceptable for critical audio. The simplest and most effective approach is a dual-switch architecture where each AES67-capable device connects to two independent network fabrics—commonly called Network A and Network B. These fabrics should be physically separate, using different switches, power feeds, and cable paths. For Layer 2 redundancy, Rapid Spanning Tree Protocol (RSTP) or Multiple Spanning Tree Protocol (MSTP) can be configured to provide failover within a few seconds, but that may be too slow for live audio. Many broadcasters opt for Link Aggregation Groups (LAGs) to protect against link failures, though LAGs do not handle whole-switch failures.
The gold standard for path redundancy in AES67 is SMPTE ST 2022-7, which defines seamless protection switching for RTP streams. The source transmits two identical streams (primary and backup) over independent network paths. The receiver aligns them using RTP sequence numbers and timestamps and switches to the redundant stream with zero audible gap upon packet loss or path failure. Because the failover decision happens at the endpoint, ST 2022-7 eliminates dependence on network convergence protocols. When combined with dual fabrics, it provides hitless protection against most network faults, including switch failures and cable cuts.
Device Redundancy
Audio interfaces, preamplifiers, and processors must also be redundant. An N+1 configuration—where multiple devices share the load and a spare takes over on failure—is common. For example, a mixing console might use two separate AES67 interface cards, each homed to a different network switch. Similarly, critical microphone signals can be split to two preamp units, each generating its own AES67 stream. With ST 2022-7, the primary stream originates from Device A and the backup from Device B, protecting against source device failure.
Power supply redundancy is equally vital. All active network components—switches, PoE injectors, media converters—should have dual, hot-swappable power supplies connected to separate UPS feeds. Professional-grade switches often feature redundant fans and N+1 power modules to prevent a single component failure from taking down an entire segment.
Failover Mechanisms
Automated failover must be fast and deterministic. For Layer 3 networks, equal-cost multipath (ECMP) routing with OSPF or static routes provides near-instantaneous rerouting when a link or router fails. For multicast streams, PIM Sparse Mode with an IGMP querier ensures that redundant paths are available without loops. Precision Time Protocol (PTP) requires its own redundancy: deploy at least two PTP grandmasters (GMs) on separate networks or VLANs. Using the Alternate BMCA (Best Master Clock Algorithm) defined in IEEE 1588-2008, slave clocks can fall back to a secondary GM when the primary fails. Ensure all endpoints support clock class fallback so that failover is seamless and within the jitter buffer tolerance (typically 1–5 ms).
Resilience Beyond Redundancy
Redundancy alone does not guarantee resilience. A network with dual paths can still suffer from packet drops due to congestion, misconfigured QoS, or timing drift. The following design areas ensure the network can withstand stress and maintain audio integrity.
Quality of Service (QoS) and Traffic Shaping
AES67 streams are time-sensitive and require low jitter. On converged networks carrying data, video, or control traffic, audio packets must be prioritized. Use IEEE 802.1p priority tagging with a DiffServ Code Point (DSCP) of 46 (Expedited Forwarding) for AES67 traffic. Configure strict priority queuing on all switches so that audio is transmitted before best-effort traffic. Ingress policing should be enabled to prevent bursty non-audio flows from overwhelming the audio queues. Critically, both primary and backup switches must enforce identical QoS policies to maintain consistent latency during failover.
Network Segmentation with VLANs
Isolate AES67 traffic into its own VLAN to reduce broadcast traffic and improve security. Use IGMP snooping on switches to forward multicast streams only to ports that have joined the relevant group. This prevents flooding and reduces CPU load on endpoints. In a dual-network topology, assign each physical network to a separate VLAN or IP subnet to avoid loops while keeping traffic logically distinct. A well-proven approach is the three-VLAN model: one VLAN for audio media, one for control and configuration, and one for PTP timing. This separation ensures that heavy data streams do not interfere with synchronization.
Timing Resilience (PTP Redundancy)
Precision Time Protocol (PTP) is the backbone of AES67; loss of synchronization causes receiver buffer overflows or underflows, producing audible glitches. Deploy at least two PTP grandmasters on independent power and network paths. Use the Alternate BMCA to allow automatic failover. All AES67 endpoints should support “clock class fallback” or “stateless” mode for redundant GMs. Test the failover time—it must be shorter than the audio jitter buffer depth (typically 1–5 ms). For additional resilience, consider using PTP boundary clocks in the network that can hold timing through short disruptions.
Power and Cable Diversity
Connect each power supply of dual-supply components to separate UPS and generator feeds. Route primary and backup Ethernet cables through different cable trays, conduits, or building risers to protect against physical damage from construction, fire, or accidental cuts. Document all cable runs and ensure that maintenance personnel are aware of the diversity requirements.
Implementing Redundancy in Practice
Translating theory into a working system requires a methodical approach. The following steps outline a practical deployment sequence for a redundant AES67 network.
Step 1: Assess Requirements and Constraints
Document the number of audio channels, maximum acceptable latency (e.g., 1 ms for live monitoring), and availability targets (e.g., 99.999% uptime). Determine whether the application can tolerate a short audio gap (under 10 ms) or requires hitless protection. This decision drives the choice between fast-converging Layer 2 protocols versus SMPTE ST 2022-7. Also consider budget: dual fabrics and redundant endpoints increase cost but may be mandatory for critical services.
Step 2: Design the Topology
Create a logical diagram showing two independent network fabrics—Network A and Network B. Each fabric should be a non-blocking switch infrastructure with redundant uplinks. For larger installations, use a spine-leaf topology with at least two spine switches. Place the two fabrics in separate physical locations (different equipment rooms or floors) to protect against localized events like water leaks or power outages. For stream redundancy per ST 2022-7, the sender must generate two streams (unicast or multicast) that traverse distinct paths to the receiver. Ensure the receiver can handle both streams simultaneously and select the active one.
Step 3: Select Suitable Hardware
Choose managed switches with non-blocking backplane, sub-10 µs latency, and support for IGMP snooping, PTP boundary clock, IEEE 802.1p, and DSCP. For ST 2022-7, switches must handle the additional bandwidth of duplicate streams (about 2× the total audio bandwidth). Broadcast-grade switches such as Cisco Catalyst 9300, Arista 7280R, or Netgear M4500 series are commonly used. For endpoints, select AES67 I/O devices that offer dual network ports and native ST 2022-7 seamless protection switching. Many Ravenna and Dante (with AES67 mode) devices support these features.
Step 4: Configure Network Redundancy
If using spanning-tree, set root bridge priorities and tune forward delay and max age to achieve sub-second convergence. For Layer 3 redundancy, configure VRRP or HSRP on router interfaces and use OSPF or static routing with ECMP. For multicast, enable PIM Sparse Mode and configure an IGMP querier. For ST 2022-7, configure the sender to transmit two flows to different multicast group addresses (or different unicast destinations) that are routed via the two fabrics. On the receiver, enable seamless protection switching and set the buffer to accommodate path delay differences (typically 0.5–5 ms). Ensure all PTP slaves are configured to fall back to a secondary grandmaster.
Step 5: Test Failover Scenarios
After configuration, perform rigorous testing. Pull a network cable from the primary switch while monitoring the audio output via a console or RTP analyzer. Measure the audio interruption time. Repeat for power loss on a switch, PTP grandmaster failure, and link failure on an endpoint. For ST 2022-7, verify that the backup stream becomes active within the RTP sequence number offset and that no audio glitch occurs. Stress-test the network with high traffic loads to ensure QoS remains intact. Document all test results and adjust buffer sizes or QoS parameters as needed. Conduct these tests quarterly to catch any degradation over time.
Monitoring and Maintenance
Redundancy is not a set-and-forget feature; it requires continuous monitoring. Implement network management platforms that collect SNMP traps and streaming telemetry (sFlow, NetFlow). Set alerts for high CPU utilization on switches, excessive packet drops, or PTP clock offset exceeding 1 µs. Many facilities use dedicated AoIP monitoring tools such as DADman (for Ravenna) or Dante Controller (when operating in AES67 mode) to visualize stream health. Additionally, MQTT-based dashboards can aggregate PTP status from multiple grandmasters. Schedule quarterly disaster drills where active network paths are intentionally taken offline to confirm that backup systems engage cleanly. Review IGMP snooping logs to identify unauthorized multicast groups that may consume bandwidth. Finally, keep detailed documentation of switch configurations, PTP profiles, VLAN mappings, and stream redundancy settings. As firmware updates and new devices are added, update the documentation and retest.
Conclusion
Designing a redundant and resilient AES67 network for critical broadcast applications is a multi-layered endeavor that goes beyond simple dual paths. It requires selecting appropriate hardware, implementing stream-level protection via SMPTE ST 2022-7, and ensuring that timing, QoS, and monitoring are all aligned. The investment in dual switches, redundant interfaces, and thorough testing pays off by eliminating downtime during live events, emergency broadcasts, or routine maintenance. Ultimately, a well-engineered AES67 network gives broadcasters the confidence that their audio will reach the audience reliably, even when failures occur. By adhering to these principles—and coupling them with proactive operations—engineers can deliver the high availability that modern broadcast demands.
For further reading, consult the official AES67 standard and the SMPTE ST 2022-7 specification. Practical guidance on PTP timing in AoIP networks is available from the NTP project and from manufacturers’ deployment guides such as those from Audinate and MediaTek.