This thesis focuses on a complete color image processing sensor. Lane recognition and object detection are the main tasks to be accomplished by the sensor. It is integrated in a sensor fusion environment in use at Group Research, Volkswagen AG together with radar and lidar sensors. The hardware used is a color camcorder linked to a processing unit (PC) via firewire. The chosen approach is to create an internal representation of the environment which is updated between aquired images. Based on this representation the required 3D output is generated, taking into consideration the camera calibration data. I have tried to approach the subject of image processing in automotive applications using two new directions. One is real time color image processing, where a new segmentation method was introduced. The other is a cognitive approach, in which knowledge over the current situation is used to merge the results of more relatively simple algorithms. The new methods introduced in this work put a basis for a large development direction that should contribute to establishing color image processing in the automotive field.