The attack, in the town of Kawasaki, south of the Japanese capital, also injured 17 people on Tuesday, local emergency officials told AFP.
The rampage was a rare attack in a country with one of the lowest rates of violent crime in the developed world, and there was no immediate detail on the motive of the knifeman.
It came as Donald Trump wraps up a state visit to Japan, and the US president offered his “prayers and sympathy” to the victims as he met troops outside Tokyo.
The attack occurred during the busy early morning commute as workers headed to their offices and children to school in Kawasaki.
Local media said the attack killed a schoolgirl and that the suspected attacker, who was taken into custody by police, had died after stabbing himself.
“One man and one female child are showing no vital signs,” fire department official Yuji Sekizawa had earlier told AFP, employing a phrase commonly used in Japan to mean the victims have died but the death has not yet been certified by an official medical professional.
The fire department said children were also among the 17 others injured.
The scene of the attack was swarming with emergency personnel, including police who had parked three police vans around the site to block it from view.
A white school bus with blue stripes could be seen parked, however, with the name “Caritas Gakuen (school)” written on the side, an AFP correspondent at the scene said.
Emergency workers had also set up an orange medical tent to treat the wounded, and ambulances and fire engines were parked nearby.
An official at a local hospital said it had received five people wounded in the attack and that four of them had suffered serious injuries.
Among them were a woman, a man and “three girls who are all six years old.”
Police earlier told AFP one suspect had been detained, and an eyewitness told local media that the man had stabbed himself.
“I saw a man holding a knife… I couldn’t see clearly, but he apparently stabbed himself in the neck,” one eyewitness told NHK.
The broadcaster said two knives were spotted at the scene, but there was no immediate confirmation from officials.
‘Scary to see’
How the attack unfolded was not immediately clear, with some initial reports saying it occurred in a park, but subsequent accounts describing it as taking place at a nearby bus stop.
“I heard the sound of lots of ambulances and I saw a man lying near a bus stop bleeding,” a male eyewitness, who was not identified, told NHK.
“There is another bus stop near the elementary school and I also saw elementary schoolchildren lying on the ground… It’s a quiet neighborhood, it’s scary to see this kind of thing happen.”
“It is a very harrowing incident,” government spokesman Yoshihide Suga told reporters, referring all further questions on the attack to police.
Violent crime is extremely rare in Japan, and children often travel to and from schools alone, but there have been a few high-profile attacks.
In 2018, a man was arrested in central Japan after stabbing one person to death and injuring two others aboard a bullet train, an attack that prompted new security measures on the famed rail service.
And in 2016, a man stabbed 19 people to death in a disability center south of Tokyo in what he described as a mission to rid the world of people with mental illness.