Charlton had threatened to be the dark horses for an automatic spot following an impressive winning run that had seen them threaten to gatecrash the top two.
But their defeat at Oxford, coupled with wins for Luton, Barnsley, Sunderland and Portsmouth has seen those hopes dashed, believes Bowyer.
Luton lead the division with 90 points from 43 games, second place Barnsley five points behind them with Sunderland third and Portsmouth fourth, both on 83 points.
Charlton remain fifth on 79 points and are now six points off the top two with only three games to go.
Sunderland suffer major injury blow
Newcastle United transfer news: Magpies identify £40m James Maddison alternative with ex-Crystal Palace man linked
Brendan Rodgers reveals he talked Swansea City defender out of move to Newcastle United whilst on Mount Kilimanjaro
Eddie Howe’s surprise Newcastle United transfer admission as Brighton draw reinforces priority
Newcastle United transfer target confirms move ahead of official announcement
Both Sunderland and Pompey also have games in hand on their automatic promotion rivals.
Bowyer now accepts it will be the lottery of the play-offs for his Charlton side this season.
He said: “It [the top two] is done. That’s done, unless they (Barnsley, Sunderland and Pompey) all go and lose, but that’s not going to happen. That’s it.
“We needed to win all four (remaining games), that’s not going to happen so it’s done.
“On another day I think we score five or six goals. I don't know how we haven't scored in that second-half.
“First half we created chances and got into good areas, so many times.
“I’ve just said to them “you’ve got to learn from that because we can’t keep getting into those good areas and not having the end product”.
“That’s cost us, switching off on set-pieces.”