Second-half goals from Ritsu Doan and Ao Tanaka overturned Alvaro Morata's opener for Spain.
What can indeed be described as a 'game of two halves' saw Spain dominate the first 45 minutes before an incredible Japanese turnaround saw them score twice and then hang on at the end as Spain looked for an equalizer that would have taken themselves to the top of the group while knocking Japan out.
Morocco Progress Into World Cup Knock-Out Stage After 36 Years
A nerve-wracking second half saw all four teams from the group in a qualifying spot at one moment or another, and for a nerve-shredding couple of minutes, even the Spanish faced an early exit.
Japan will face Croatia in the round of 16, while Spain will face Morocco.
As things got underway two hours earlier, there was no indication of the drama ahead.
Spain coach Luis Enrique made five changes to the team that drew 1-1 with Germany on Sunday, with Alejandro Balde, Nico Williams, Cesar Azpilicueta, Pau Torres and Morata coming into the side, while Shogo Taniguchi, Junya Ito, Tanaka, Takefusa Kubo and Daizen Maeda were in for Japan.
After Ito's shot nearly punished a rare lapse from Sergio Busquets into the side netting, Spain took total control, racking up around 80 percent of ball possession and leaving their rivals chasing shadows.
Morata scored his 30th international goal in 60 appearances with a neat header after Williams and Azpilicueta combined after 11 minutes and forced a save from Shuichi Gonda after running onto Pedri's diagonal ball.
Spain continued to dominate, with every possession lasting an average of over a minute and only Unai Simon's work with his feet causing pulse-rates to flutter.
It looked as if a Japan side that had seen just 14 percent of ball possession in the first half was going out with barely a whimper. Still, coach Hajime Moriyasu replaced Kubo and Yuto Nagatomo with Doan and Kaoru Mitoma, and turned the game on their head.
The subs made an immediate impact, with Doan robbing Balde of possession and lashing in a left-footed shot past Simon to level the score in the 48th minute and then putting the ball across the goal in the 51st minute.
Mitoma just kept it in play and Tanaka scrambled home from close range, with the goal given after a lengthy VAR check.
Suddenly Japan was on a different side: fast and aggressive and causing Spain all sorts of problems, with Mitoma producing a great run and cross only for Takuma Asano to slip at a vital moment.
News of Costa Rica taking the lead against Germany, which momentarily saw Spain eliminated, and Germany's subsequent equalizer flashed quickly, raising the tension to almost unbearable levels.
Gonda saved from Marco Asensio in the 88th minute, and as the defense scrambled the loose ball clear, he then got down to save from Torres.
The seven minutes of extra time saw Spain camp on the edge of the Japan penalty area, looking for a goal that would see them top the group and knock out Japan.
However, Japan held on for an incredible win and sent four-time winners Germany back home.