There was no word yet on the fate of the one pilot on board the jet, Defense Minister Takeshi Iwaya said.
“We have collected part of its tail” in search operations at sea with planes and vessels, Iwaya told reporters.
“We believe it crashed,” he added.
The fighter jet went missing around 7:30 pm (1030 GMT) Tuesday as it was flying some 135 kilometers east of Misawa, northeastern Japan, on a training mission.
The plane lost contact about 30 minutes after taking off from Misawa Air Base with three other aircraft.
Japan is deploying F-35As, each of which costs more than 10 billion yen (90 million dollars), to replace the aging F-4 fighter jet.
The jet was one of 13 F-35s deployed at the base, according to a defense official.
The remaining 12 F-35A fighters have been grounded for the time being, according to the Defense Ministry.