Toshiro Muto said he would keep an eye on infection numbers and hold "discussions" if necessary.
More than 70 people associated with the Games have tested positive ahead of Friday's opening ceremony.
His comments came on the same day International Olympic Committee (IOC) president Thomas Bach said cancellation was "never an option".
Earlier this month, Japan announced the Games would go ahead in empty stadiums - despite growing concern over the Covid situation.
"We will continue discussions if there is a spike in cases," said Mr Muto when asked at a press conference if the Games might be cancelled even at this late stage.
"At this point, the coronavirus cases may rise or fall, so we will think about what we should do when the situation arises."
Although the Games don't officially begin until Friday, the sporting action has kicked off with the women's softball and football competitions starting on Wednesday.
Earlier this week, two South African footballers staying in the athletes' village tested positive for coronavirus. A Czech beach volleyball player later tested positive for the virus.
Read More (...)