In a frantic opening half, Jose Baxter gave Sheffield United a shock lead on 19 minutes.
Yannick Sagbo equalised for Hull on 42 minutes, but Stefan Scougall restored the League One side’s advantage just before halftime.
Hull fought back at the start of the second period with substitute Matty Fryatt and Tom Huddlestone scoring in quick succession, before Stephen Quinn’s header effectively wrapped up victory for the Premier League side.
Jamie Murphy snatched a late consolation for United in injury time, before Hull added a fifth through David Meyler.
Hull will now play Arsenal in the final at Wembley on May 17 after Arsene Wenger’s side sneaked past Wigan Athletic on penalties in the other semi-final on Saturday.
After being run so close by the League One side, Hull manager Steve Bruce was both delighted and relieved to reach the final.
“We were awful first half, but I have to say a special well done to Sheffield because a League One team were terrific,” he told reporters.
“They handled it (the occasion) better than we did in the first half so all credit to them.
“But I am delighted for Hull City to play Arsenal in an FA Cup final.”
Sheffield United manager Nigel Clough was left to rue the equalising goal they conceded after halftime.
“We were close to making history as the first third-tier side to get to an FA Cup final and at halftime I thought we were well worthy of our lead,” he said.
“We played well first half and they changed it second half. The second goal, the one just after halftime, was the killer blow for us as it got them totally going.”
Sheffield United looked comfortable against their top-flight opponents in the opening stages, passing the ball crisply round the Wembley turf before taking the lead on 19 minutes when John Brayford’s cross was prodded in by former Everton forward Baxter.
Hull rarely troubled United for much of the first half, but they snatched an equaliser when Sagbo latched on to Jake Livermore’s perfectly-weighted pass to fire the ball into the roof of the net.
The equaliser was harsh on United, who were playing with plenty of vim and vigour, and they restored their advantage two minutes later when Murphy tore down the left flank and cut the ball back from the byline for Scougall to lash past Steve Harper.
In search of more attacking impetus, Hull manager Steve Bruce introduced forwards Sone Aluko and Fryatt at halftime, and it paid off as United failed to deal with a corner and Fryatt was on hand to sweep the ball home.
United were now on the back foot and a neat one-two between Meyler and Huddlestone on 54 minutes enabled the former Tottenham Hotspur man to burst into the box and place a left-footed shot past Mark Howard, giving Hull the lead for the first time.
With half an hour remaining substitute Quinn, a former Sheffield United player, headed in Livermore’s chipped cross to give the Premier League side a bit of breathing space.
But United, bidding to become the first third-tier side to reach an FA Cup final, refused to roll over and when Murphy smacked a loose ball into the net it set up a nervy few minutes for the Hull supporters.
With Nigel Clough’s side throwing men forward at will in the push for an equaliser, they were caught on the break by Hull and a sweeping move upfield enabled Meyler to apply the finish and send the Premier League side into the final.
(Editing by Toby Davis) nL3N0N508H