Abstract: Action recognition is one of the major application areas of computer vision and focuses more on temporal dynamics than static image classification. Recently, many lightweight temporal modeling methods based on 2D convolutional neural networks (CNN) have been proposed for action recognition. However, these models often fail to effectively capture long-range temporal dependencies in motion sequences. To address this limitation, an enhanced temporal multimodal attention network (E-TMAN) is proposed to model temporal features through a two-stage architecture. E-TMAN consists of two main modules: local motion enhancement (LME) and difference time interaction (DTI). The LME module employs a temporal attention mechanism to fuse temporal information into spatial features, thereby enhancing the representation of short-term motion patterns. The DTI module adopts a differential learning strategy to capture long-term temporal dependencies through two complementary attention mechanisms, namely channel attention and spatial attention. By integrating temporal and contextual information in different forms, the proposed network achieves more effective temporal feature modeling. Extensive experiments are conducted on three public benchmarks: SomethingV2, UCF101 and HMDB51. The results show that E-TMAN effectively exploits temporal features and achieves competitive performance with lower computational cost compared with state-of-the-art methods.