Object detection is one of the most important and challenging problems in the field of computer vision. In the current mainstream detection approaches, especially in the architectures of feature pyramid network (FPNs), feature fusion is a basic and essential method for all detectors. However, feature fusion does not fully consider the characteristics of the detection task for most detectors. To obtain suitable features for the detection task, in this paper, we propose two fusion methods: (1) For feature extraction, we propose an improved feature pyramid network (ImFPN) for superior representations. The most essential difference from FPNs is that the ImFPN includes a similarity-based fusion module, which can fuse different features to adapt to varying sizes of instances. (2) For specified tasks, since classification and regression tasks have different considerations in the same region, we build a new fusion mechanism between the dense and sparse heads in any two-stage detector based on an improved region proposal network (ImRPN). After adding these two modified architectures to Faster R-CNN with ResNet-101, the average precision (AP) improves from 39.7 to 41.4 on COCO test-dev . In addition, extensive experiments show the effectiveness of our methods on various models and datasets.