In this paper, we introduce a novel fast object detection framework, designed to meet the needs of real-time applications such as autonomous driving and robot navigation. Traditional processing methods often trade-off between accuracy and processing speed. To address this issue, we propose a hybrid data representation method that combines the computational efficiency of voxelization with the detail capture capability of direct data processing to optimize overall performance. Our detection framework comprises two main components: a Rapid Region Proposal Network (RPN) and a Refinement Detection Network (RefinerNet). The RPN is used to generate high-quality candidate regions, while the RefinerNet performs detailed analysis on these regions to improve detection accuracy. Additionally, we have implemented a variety of network optimization techniques, including lightweight network layers, network pruning, and model quantization, to increase processing speed and reduce computational resource consumption. Extensive testing on the KITTI and the NEXET datasets has proven the effectiveness of our method in enhancing the accuracy of object detection and real-time processing speed. The experimental results show that, compared to existing technologies, our method performs exceptionally well across multiple evaluation metrics, especially in meeting the stringent requirements of real-time applications in terms of processing speed.