Wrong.
Everything in the Bible "matters" in some way -- It's not unimportant just because you don't see why.
And what day would that be?
Immediately after the fig tree incident, Jesus arrives in Jerusalem, to a crowd waving leafy branches and shouting "Hosanna," correct? Now, the problem is that's not typically what Jews do for Passover... however, during the Feast of the Tabernacles (Sukkot), Jewish pilgrims would march around the altar of the Temple, waving leafy branches, and chanting Psalm 118... proclaiming "Hosanna."
Now, Sukkot falls in the Jewish month of Tishrei, not Nisan -- usually around mid to late September, when one would expect the figs to be in season.
Question: Why would Jesus show up in Jerusalem in time for one holiday to a crowd celebrating in the tradition of a different one? It would be like him arriving on St. Patrick's Day to a crowd chanting "Trick or Treat."
The more you look at the fig tree story, the more it seems so out of character that it's bound to attract attention... almost as if it's warning the reader that something is amiss...