In this project, semantic labeling for facade objects and roof components from 3D point clouds (acquired by mobile laser scanning) will be conducted at first, in order to prepare the training data for a deep learning based approach to learn facade and roof grammar. Together with the ongoing work of collecting rules both on 2D and 3D in urban area, a new pipeline based on convolutional neural network (CNN) will be developed by integrating the explicit urban rules. This new pipeline can serve as a solution for automated reconstruction of facade models with more promising detection and prediction. Another very important reason of developing such a pipeline is that implicit urban rules are expected to be parsed by an inverse inspecting process of the deep learning network. This could help achieving the transparency in the deep learning. At the same time, it could help understanding how semantics, geometries and topology of urban objects are related to each other in a complex environment. The output of this project will on the one hand support a revised ERC CoG proposal submission, on the other hand, the training data will be made public available as an international benchmark for semantic object detection, to enhance the international collaborations with international partners.
Project leader: Hongchao Fan