Grays suffered an early set back as David East picked up an injury after only seven minutes and had to be replaced by Ryan Kirby.
Barking took full advantage to take the lead in the 8th minute as Grays were still trying to get organised at the back.
A free kick from the left wing was played deep to the back post where Nathan Phillip was allowed to run in unmarked and head the ball into the corner.
Grays struggled to gain any momentum in the first half hour and where reduced to a couple of weak shots from the edge of the area by Kenny Beaney and Harry Agombar, both which were easily saved by Christian Baffour in the Barking goal.
Harry Agombar should have levelled the game in the 32nd minute as Luke Marshall crossed from the right to the back post where Agombar was left unmarked. From 5 yards out he had a simple chance to head home but tried to volley in with the side of his foot and fired high over the bar.
Another chance went to the away side in the 37th minute as a corner from the left was swung into the area along the six yard line where Junior Baker rose highest to head towards goal but Baffour did well to tip the ball over the bar.
Barking almost took a two goal lead into the break as in injury time of the first half; a long ball over the top of the Grays defence was chased down by Dumebi Gb-Dukaka who got to the ball before Lamar Johnson to head towards goal, but Junior Baker chased back to clear the ball before it crossed the line.
If Grays fans though the first half was bad, the first thirty minutes of the second half were even worse.
Grays replaced Leon Lalite with Jack West in the 51st minute with little effect to the game.
However, the decision to replace Jake Hall with Danny Bunce after 68 minutes proved to be the inspirational substitution that manager Hakan Hayrettin was searching for.
In the 80th minute Grays drew level Harry Agombar found space on the edge of the area before playing the ball to Danny Bunce who arrived on the corner of the area to fire first time low into the far corner of the net, much to the relief of the Grays fans.
Two minutes later Grays turned the game on its head as a cross from the right by Harry Agombar was met by the head of Danny Bunce on the penalty spot and his looping header found the top right hand corner of the net to give Grays an unlikely lead.
With Barking deflated to be behind after such a good effort, Grays charged forward in the final seconds and should have got a third goal as Harry Agombar played in Joe Sweeney but he fires wide from close range.
The final whistle went shortly after and Grays now face a Semi-Final away to Concord Rangers, who a flying high, near the top of the Ryman Premier.
Grays Athletic: Lamar Johnson, Craig Pope, David East (Ryan Kirby 7), Junior Baker, Lee Flynn, Luke Marshall, Kenny Beaney, Leon Lalite (Jack West 51), Jake Hall (Danny Bunce 68), Harry Agombar, Joe Sweeney
Unused Subs: Jared Small, Daniel Subuola