Gamma-ray bursts (GRBs) are intense, short-lived bursts of gamma-ray radiation observed up to a high redshift (z 10) due to their luminosities. Thus, they can serve as cosmological tools to probe the early Universe. However, we need a large sample of high-z GRBs, currently limited due to the difficulty in securing time at the large aperture telescopes. Thus, it is painstaking to determine quickly whether a GRB is high-z or low-z, which hampers the possibility of performing rapid follow-up observations. Previous efforts to distinguish between high- and low-z GRBs using GRB properties and machine learning (ML) have resulted in limited sensitivity. In this study, we aim to improve this classification by employing an ensemble ML method on 251 GRBs with measured redshifts and plateaus observed by the Neil Gehrels Swift Observatory. Incorporating the plateau phase with the prompt emission, we have employed an ensemble of classification methods to unprecedentedly enhance the sensitivity. Additionally, we investigate the effectiveness of various classification methods using different redshift thresholds, z = z at z = 2.0, 2.5, 3.0, and 3.5. We achieve a sensitivity of 87% and 89% with a balanced sampling for both z = 3.0 and z = 3.5, respectively, representing a 9% and 11% increase in the sensitivity over random forest used alone. Overall, the best results are at z = 3.5, where the difference between the sensitivity of the training set and the test set is the smallest. This enhancement of the proposed method paves the way for new and intriguing follow-up observations of high-z GRBs.