I have anxiety disorder, too...alongside depression with adjustment disorder (a VA term that basically means I don't make a very good civilian) and PTSD.
The first three manifest by avoidance and self-isolation, meaning I stay away from crowds and noisy venues, etc. The PTSD, however, when it springs into action, is a little more dramatic. Any type of loud, unexpected, percussive noise---especially if it resembles gunfire---makes me jump, duck, and hyperventilate. But any loud noise will do; I have been known to go into episodes when somebody near me sneezes. These usually leave me breathing rapidly, sweating lightly, and feeling shaky. The worst ones cause an amygdala hijack, which means my brain just takes over and reacts instinctively; when that happens, I end up diving for cover, and am totally incapacitated for several minutes. In those episodes, I end up hyperventilating, sweating heavily, shaking badly, my heart trip-hammers, and I am left feeling weak and woozy. I may or may not start weeping.
I was in a supermarket one time, looking at the soda pop, when a woman off to my left lost her grip on a case of Coca-Cola. It crashed into her shopping cart with a resounding boom, and I dove for the floor, spouting a profane curse, and ended up huddled against the wall, shaking and unable to function terribly well. A crowd gathered fairly quickly, and my wife, who was down the aisle a ways, came back over to me to direct traffic. I remember someone asking, "Is he okay?", and she replied, "He will be, in a little while." I came back to myself after a bit, and was able to stand up and communicate. The crowd dispersed
(show's over, folks; nothing to see here) except for the lady who had dropped the case of soda to begin with, who stood off to the side with tears running down her face because she felt responsible. I went over to her and said, "Now look, dear: it is not your fault that I have this condition, okay? So you have nothing to feel guilty about. Sometimes this just happens, and it is what it is." I smiled and gave her a hug, and hopefully she felt better. I moved off a way, and I heard my wife talking to her in a low tone; probably explaining what just happened, and everybody left on friendly terms.
Another time I was in a Lowe's or a Home Depot, can't remember which, and somebody behind us tipped over a big steel shelving unit onto the floor. It hit the concrete with a sound remarkable similar to an artillery round exploding. I remember that everything in my vision went to a grainy yellow, and when I came to, I was trying to crawl under a grocery cart. My wife said that I had yelled
"INCOMING!!!" and dove onto the floor. I was a mess after that one, crying, shaking, sweating, gasping for air. I don't remember yelling or diving, but I remember the condition I was in afterwards, quite well.
Usually they're not that bad; but they can be, in the right (or wrong) circumstances. There's nothing anybody can do about it; the VA counselors explained to me that PTSD literally re-configures parts of your brain, and when it's done, it's done. There is no cure for it; all you can do is manage the symptoms with medication or therapy. In my case, I take 100mg of Sertraline during the day to keep me on an even keel, and 50mg of Trazodone at night to kill the nightmares and let me sleep. These do me pretty well, although I still get nightmares from time to time, and I still get really twingy in crowds---tight store aisles make me extremely claustrophobic---and I usually like to sit with my back to the wall with my eye on the exit doors when we go to a restaurant.
So, I wouldn't feel too bad about ducking out early, if I were you.

Be thankful that you don't have it as bad as some others.