The paper presents an extension of the hypergraph grammar model of the hp-adaptive finite element method algorithm with rectangular elements to the case of non-stationary problems. In our approach the finite element mesh is represented by hypergraphs, the mesh transformations are modeled by means of hypergraph grammar rules. The extension concerns the construction of the elimination tree during the generation of the mesh and mesh adaptation process. Each operation on the mesh (generation of the mesh as well as h- adaptation of the mesh) is followed by the corresponding operation on the elimination tree. The constructed elimination tree allows the solver for reutilization of the matrices computed in the previous step of Finite Element Method. Based on the constructed elimination tree the solver can efficiently solve non-stationary problems.