Abstract: The storage capacity of the neural network model used for video anomaly detection is low, which leads to the lack of spatio-temporal information of feature parameter. Aiming at the research of the memory power of neural network, the Conv LSTM-spatio-temporal memory module (Conv LSTM-SMM) to compensate for the lack of storage capacity of neural network was proposed and applied to video anomaly detection based on the deep learning framework. The designed Conv LSTM-SMM consisted of Conv LSTM and SMM. The feature vectors saved by Conv LSTM were continuously updated and stored in the memory network by querying features, and the updated feature parameters were matched and stored with the query features to obtain new feature parameters to enhance the spatiotemporal features of the network. At the same time, edge learning was introduced to increase the distance between normal samples and abnormal samples. And SMM was used to guide edge learning, which was applied to video anomaly detection based on deep learning. Conv LSTM-SMM improves the memory power of feature information for anomaly detection system, which is beneficial to the integrity and accuracy of detected information. Edge learning enlarges the feature gap between normal data and abnormal data, and the combination of the two methods paves the way for obtaining high detection accuracy of video anomaly detection system. The experimental part was carried out on the most challenging public datasets (Avenue, ShanghaiTech) at present. The experimental results show that the designed video anomaly detection algorithm based on Spatio-temporal memory guided edge learning has improved the detection rate of video anomaly, and the detection performance is superior.