The aim of this Dissertation is to present an analysis of the behavior of protocols and protection methods in optical communication systems for the ultra high bit rate networks. The demand for optical networking transmission bandwidth is increasing greatly in recent decades, mainly due to the spread of access to and the use of the Internet and new types of services, with video sharing and cloud applications. This increase in traffic has driven strongly the development of optical systems technologies that seek to ensure compliance with this demand. Several fronts of studies seeking to obtain rates from higher bits per channel (100Gbps, 400Gbps and 1Tbps), using advanced technologies such as multilevel modulation format, fibers with low loss and wider effective transmission area, hybrid EDFA/Raman amplification and digital coherent detection technologies. The maximum transmission rate in optical systems is coming out of the current 10Gbps and 40Gbps and going towards the 100Gbps, already commercially offered, and still developing 400Gbps and 1Tbps. This increase in transmission rate generates a need for evaluation of the behavior of current protection protocols and methods when applied to these new ultra-fast networks. The change to so high transmission rates may require different operating strategies, and the reason is that the effects of propagation delay alter significantly the control traffic flow.