Obstacle detection is a mandatory function for a robot navigating in an indoor environment especially when interaction with humans is done in a cluttered environment. Commonly used vision-based solutions like SLAM (Simultaneous Localization and Mapping) or optical flow tend to be computation intensive and require powerful computation resources to meet low speed real-time constraints. Solutions using LIDAR (Light Detection And Ranging) sensors are more robust but not cost effective. This paper presents a real-time hardware architecture for vision-based obstacle detection and localization based on IPM (Inverse Perspective Mapping) for obstacle detection, and Otsu's method plus Bresenham's algorithm for obstacle segmentation and localization under the hypothesis of a flat ground. The proposed architecture combines cost effectiveness, high frame-rate with low latency, low power consumption and without any prior knowledge of the scene compared to existing implementations.