"Let"

This is a good example of why people shouldn't get hung up over any particular English translation of the Bible. The English translation, of this verse in particular, is not a word-for-word translation of the original Hebrew.

The word "let" in the English translation does not appear in the original Hebrew text. Those who are studying ancient Hebrew can explain this better than I can.
But according to Strong's Concordance, the phrase in this verse "But let all those who put their trust" was all translated from a single Hebrew word (H2620) that means "to seek refuge, flee for protection, to put trust in God, confide or hope in God".

And "let them ever" [...shout for joy] comes from the single Hebrew word (H5769) that mean a long duration, continuous, antiquity, forever, everlasting, etc.

"Let them also that love" comes from a single Hebrew word (H157) that means to love another human being as a friend, a lover, or for God, etc.

How the translaters translated those sparse words into the English equivalent, I don't know. Maybe there are more significant meaning by the structure of the words in the poetic form. But I wouldn't get too hung up over an English equivalent word.