Background: There has been an increasing global prevalence of depression and other psychiatric diseases in recent years. Perceived stress has been proven to be associated with psychiatric and somatic symptoms. Some animal and human studies have suggested that consuming foods abundant in lignans and phytosterols may be associated with lower levels of stress, depression, and anxiety. Still, the evidence is not yet strong enough to draw firm conclusions. Thus, we investigated the association between dietary intake of these phytochemicals and the level of stress experienced by adult individuals. Methods: Diet was assessed using self-reported 7-day dietary records. The intakes of lignans and phytosterols were estimated using databases with their content in various food products. The Perceived Stress Scale (PSS) was implemented to measure the level of perceived stress. A logistic regression analysis was used to test for associations. Results: The odds of elevated PSS were negatively associated with dietary intake of total phytosterols, stigmasterol, and β-sitosterol, with evidence of a decreasing trend across tertiles of phytochemicals. The analysis for doubling the intake reinforced the aforementioned relationships and found protective effects against PSS for total lignans, pinoresinol, and campesterol. Conclusions: Habitual inclusion of lignans and phytosterols in the diet may play a role in psychological health. To address the global outbreak of depression and other mental health issues triggered by stress, it is important to take a holistic approach. There is a need to develop effective strategies for prevention and treatment, among which certain dietary interventions such as consumption of products abundant in lignans and phytosterols may play a substantial role.