Rolling into the office around 9:30 a.m. every day may be killing your career, even if you stay late to put in a full day’s worth of work.

A new study from the University of Washington’s Foster School of Business found that despite working the same amount of time, a worker who starts the day earlier will come off as more conscientious, and consequently receive better performance reviews, than someone who starts later.

Past studies have shown that giving workers a little leeway in setting their hoursmakes them happier and more productive — presumably a win for both the employee and the company. That’s why companies like Google and Microsoft let employees work from home or adjust their schedules to account for personal responsibilities, like taking a child to school in the morning.

But the study suggests a downside: Your boss may be unconsciously judging you for giving yourself a later start time, regardless of how much work you’re getting done.

In the study, which will be published in full in the Journal of Applied Psychology later this year, researchers asked 149 supervisors to rate an employee’s performance; employee start times ranged from 5 a.m. to 9:45 a.m. Even when employees who started later in the day worked the same number of hours as those who started earlier, supervisors were inclined to rate employees who began work earlier as more conscientious, and gave them higher performance ratings.

Bosses who preferred to work later hours themselves were less inclined to display this “morning bias.”

In a second experiment, the researchers assigned a group of 141 college students the role of a supervisor, and asked each student to review a profile of a fictitious employee. The performance of the fictional employee was the same in all cases, but start time varied across profiles. Researchers found that employees who worked from 7 a.m. to 3 p.m. were consistently given higher ratings of conscientiousness and performance by the student supervisors than employees who worked from 11 a.m. to 7 p.m.

The researchers concluded that employees who can opt for flexible work schedules should shift their schedules earlier, not later, to account for managers’ morning bias.

“It seems likely that some employees are experiencing a decrement in their performance ratings that is not based on anything having to do with their actual performance,” the three researchers wrote in a post for the Harvard Business Review.

This “morning bias” appears to be deeply ingrained. In a word-association experiment designed to test the implicit stereotype that early risers are more diligent workers, the researchers also found that a majority of the 120 working adults surveyed linked words associated with the morning (like “sunrise”) with words associated with conscientiousness (like “industriousness”).

Another recent study covered by the New York Times further suggests there may be an implicit view among some managers that employees who take advantage of flexible work policies in any way are less conscientious than employees who work more “traditional” hours.

“Most organizations still treat workplace flexibility as an accommodation,” Erin Kelly, a sociology professor at the University of Minnesota, told the Times. “You are implicitly saying, ‘Most of us will be working these traditional ways and the rewards will come to those working these traditional ways.’ And that is where you have this stigma or career penalties.”

Although the study did not consider gender, it seems women could be more adversely affected by morning biases than men. That’s because more women than men are primary caregivers for their children and assume a disproportionate amount of household tasks, according to a 2013 Bureau of Labor Statistics report. Studies have shown that women take advantage of workplace telecommuting policies more often than men for the same reason — and that their careers may suffer as a result of not logging as much face time at the office.