In this work we analyze the structure of the thermodynamic arrow of time, defined by transformations that leave the thermal equilibrium state unchanged, in classical (incoherent) and quantum (coherent) regimes. We note that in the infinite-temperature limit, the thermodynamic ordering of states in both regimes exhibits a lattice structure. This means that when energy does not matter and the only thermodynamic resource is given by information, the thermodynamic arrow of time has a very specific structure. Namely, for any two states at present there exists a unique state in the past consistent with them and with all possible joint pasts. Similarly, there also exists a unique state in the future consistent with those states and with all possible joint futures. We also show that the lattice structure in the classical regime is broken at finite temperatures, i.e., when energy is a relevant thermodynamic resource. Surprisingly, however, we prove that in the simplest quantum scenario of a two-dimensional system, this structure is preserved at finite temperatures. We provide the physical interpretation of these results by introducing and analyzing the history erasure process, and point out that quantum coherence may be a necessary resource for the existence of an optimal erasure process.