Performance Optimisation and Simulation-Based Validation of 400 Gbps Dual-Polarization QPSK Optical Backhaul Systems for 5G and Beyond Mobile Networks

We present a comprehensive simulation-based optimisation framework for 400 Gbps dual-polarisation quadrature phase-shift keying (DP-QPSK) optical backhaul systems targeting 5G and 6G mobile networks. The framework integrates the nonlinear Schrödinger equation (NLSE), the Manakov equation, and practical hardware constraints to model signal propagation under chromatic dispersion (CD), polarisation mode dispersion (PMD), Kerr nonlinearities, and amplified spontaneous emission (ASE) noise. Using commercially available erbium-doped fibre amplifiers (EDFA, NF = 5 dB) and standard single-mode fibre (SSMF), our optimised design achieves 1200 km unrepeated transmission at 4 dBm launch power, operating 4.7 dB above the standard quantum limit (SQL). The proposed digital signal processing (DSP) chain employs an adaptive step-size split-step Fourier method (SSFM), intelligent sampling with 8× reduction, and a hybrid CNN-BiLSTM equaliser that reduces computational complexity by 93% relative to simplified digital backpropagation while maintaining a pre-forward-error-correction (pre-FEC) bit error rate (BER) below 2.4 × 10⁻². Monte Carlo simulation with 10⁶ trials confirms a 2.3 dB Q-factor improvement over standard DSP, a 60% reduction in phase noise, and 35% lower power consumption. A stochastic-geometric channel model improves reach prediction accuracy by 25% compared to the Gaussian noise model. An ablation study confirms that each component of the proposed architecture contributes statistically significant performance gains (p < 0.01). A feasibility analysis of FPGA implementation indicates that the CNN-BiLSTM equaliser can operate at 32 GBaud with 128 parallel engines in 7nm CMOS, with estimated power consumption of 150 mW and latency of 2.4 µs. Comparison with commercial 400G systems demonstrates competitive cost-per-Gbps and power efficiency. Hardware-in-the-loop validation on FPGA and field trials are identified as future work. The proposed architecture provides a scalable pathway toward 600G–1T optical backhaul for 5G-Advanced and 6G networks.