The applications of digitally modulated signals are still in progress and expansion. Automatic Modulation Identification (AMI) is important to classify the digitally modulated signals of arbitrary modulation schemes. AMI is crucial in military applications, like electronic surveillance, and interference recognition. In civil applications, AMI can be employed in Software Defined Radio (SDR), signal monitoring, intelligent modems, Cognitive Radio (CR), etc.To get better results of the system suggested optimization the features to discard weak or irrelevant features in the system and keep only strong relevant features, Thus increasing the accuracy of the system in identifying the modified signals.In this work, present hybrid intelligent system for the recognition related to the digitally modulated signals where used , which include 3 major modules: optimization module, classifier module, as well as feature extraction module. The proposed (AMI) had been built to classify ten most popular schemes of digitally modulated signals, namely (2ASK, , 2PSK, 4PSK, 8PSK, 8QAM,16QAM ,32QAM, 64 QAM, 128QAM, and 256QAM), with the signal to noise ratio ranging from (-2 to 20) dB. High-order cumulants (HOCs) as well as high-order moments (HOMs) were utilized. The parameters of features extraction were minimized by decreasing the numbers of HOMs and HOCs, in order to reduce the complexity and training time of the AMI systems using modified optimization techniques .In this thesis used two optimization algorithms Chicken Swarm Optimization (CSO) , Bat Swarm Optimization (BA) to optmize.The Random Forest ( RF) classifier was introduced for the first time In this thesis. Simulation results of the System proposed , under additive white Gaussian noise channel show that when used (RF-CSO) have 95 % success rate for SNR between (20-13 )dB and the a Classification accuracy of (94%) for the SNR ranging from (6-12) dB for the low SNR values (5≥ SNR) .The classification accuracy (90%) dB .While algorithm ( BA Swarm Optimization ) for the modulated signals obtained a classification accuracy of around 90% for the SNR between (20-13 ) dB and the a classification accuracy of (92%) for the SNR classification from (6-12) dB for the low SNR values(5≥SNR) The classification accuracy (89%) . A fair comparison was carried out between the two system proposed to feature extraction optimization and with the best classifier under the same circumstances. The comparison clearly showed that the used (RF -CSO) have higher success rate than those of the (RF-BA).