Wolves will smash their transfer record when the Portuguese winger makes his loan move from Benfica permanent this week in a £13m deal.
Wolves moved to tie Costa up after he set up first-half goals for Richard Stearman and Andreas Weimann before Divock Origi’s 86th-minute consolation.
Costa’s heroics saw the Championship side reach the fifth round for the first time since 2008 to leave their 8,300 fans delirious.
The nine-goal top scorer also stunned Anfield with one 70-yard run before firing wide in between his assists.
Stearman’s goal after 52 seconds was his first in 95 games while Weimann’s first since joining on loan from Derby came on his full debut, the 200th appearance of his career, three minutes before half-time.
Stearman, 29, said: “I don’t see why Costa can’t perform at this level. On performances like that, he’s a special player so hopefully he does and it’s with Wolves.
“Hopefully he will be a Wolves player for a while now. He was incredible.
“As soon as it left his foot I knew I was in with a chance of scoring.
“To score at Anfield and win in front of the Kop is incredible.”
Weimann, 25, played with Ashley Young, Fabian Delph, Christian Benteke and Charles N’Zogbia at Villa.
But he rates Costa as good as any of them. He said: “Going by that performance he’s definitely up there with the best I have played with.
“He can carry the ball and run past people with ease, he’s brilliant, amazing.”
Wolves thoroughly deserved their famous win and could have grabbed a third but sub Jon Dadi Bodvarsson toe-poked at Loris Karius and had a shot blocked.
Liverpool’s only other real threat came when keeper Harry Burgoyne, 20, kicked Origi’s shot off the line.
Wolves boss Paul Lambert has never lost in five trips to Anfield with Celtic, Norwich, Villa and Wolves.
The Scot, 47, said: “My lads will never forget that day and they will be revered for coming to Anfield and winning 15, 20 years down the line.”
Liverpool have now lost three games in a row – all at Anfield – and crashed out of two cups in a week.
They host Premier League leaders Chelsea tomorrow 10 points adrift in fourth place.
Midfielder Philipps Coutinho said: “It has been a bad week, but we are looking beyond that.
“Of course we believe we can win titles under this manager – we are confident that will happen.”
Klopp made nine changes to the side that lost the EFL Cup semi-final to Southampton.
He said: “I have nothing good to say about this game apart from it is over.
“This makes the pressure even more. But it is about winning games and we have the chance again.”
LIVERPOOL (4-3-3): Karius 4; Randall 4 (Coutinho 46, 6), Gomez 5, Klavan 6, Moreno 5; Wijnaldum 5, Lucas 6, Ejaria 5 (Can 74); WOODBURN 7; Firmino 5 (Sturridge 65, 5); Origi 5.
Subs: Mignolet, Milner, Stewart, Wilson.
UP NEXT: Chelsea (h), Premier League, tomorrow
WOLVES (4-3-3): Burgoyne 8; Coady 8, Stearman 9, Hause 9, Doherty 8; Evans 8, Saville 8, Edwards 9; COSTA 9 (Ronan 66, 7), Dicko 8 (Bodvarsson 76, 8), Weimann 9 (Mason 76, 7).
Subs: Lonergan, Batth, Enobakhare, Gibbs-White.
UP NEXT: Barnsley (a), Championship, tomorrow
Referee: Craig Pawson 7
