This paper continues development of ontological approach to conceptual visual design aided by computer. Design ideas during the conceptualization phase are externalized by the designer in the form of diagrams on the monitor screen and automatically transformed by the system into data structures being hyper-graphs. Hyper-graph structures are combined with logic-based knowledge representation techniques. Different types (sorts) are used to represent knowledge from diagrams and many-sorted first order languages for their formal specification. The paper is the next attempt to formalize ontology-based knowledge framework for CAD process. The proposed method is illustrated with an exemplary of design of floor-layouts aided by the prototype of the System, so called HSSDR (Hyper-graph System Supporting Design and Reasoning).