YCGP, here is a sample of Patristic witnesses, both Antiochian and Alexandrian, Eastern and Western, that show you how those speaking from within the Church deal with literal/history and symbolism in Genesis:
1. St. Methodios of Olympos, Concerning Chastity 3.2:
For it is a dangerous thing wholly to despise the literal meaning, as has been said, and especially of Genesis, where the unchangeable decrees of God for the constitution of the universe are set forth, in agreement with which, even until now, the world is perfectly ordered, most beautifully in accordance with a perfect rule, until the Lawgiver Himself having re-arranged it, wishing to order it anew, shall break up the first laws of nature by a fresh disposition. But, since it is not fitting to leave the demonstration of the argument unexamined-and, so to speak, half-lame-come let us, as it were completing our pair, bring forth the analogical sense, looking more deeply into the Scripture; for Paul is not to be despised when he passes over the literal meaning, and shows that the words extend to Christ and the Church.
2. St. Cyril of Alexandria, Commentary on the Prophet Isaiah 1.4, PG 70.192AB
Those who reject the historical meaning in the God-inspired Scriptures as something obsolete are avoiding the ability to apprehend rightly, according to the proper manner, the things written in them. For indeed spiritual contemplation is both good and profitable; and, in enlightening the eye of reason especially well, it reveals the wisest things. But whenever some historical events are presented to us by the Holy Scriptures, then in that instance, a useful search into the historical meaning is appropriate, in order that the God-inspired Scripture be revealed as salvific and beneficial to us in every way.
3. St. Nilus of Sinai, Letter 2.223, PG 79.316BC
If something has been recorded in the Old or New Testament to have happened historically, and this or that deed was manifestly accomplished, and we interpret it for our own purposes, using ideas and thoughts for our own spiritual edification, do not suppose that we have disregarded the letter, or rejected the history. By no means! We neither condemn nor reject the perceptible event that has been committed to history. Since, however, we are [in] the world, we benefit today by interpreting everything that happened yesterday for our own purposes.
4. St. Augustine, City of God 13.21
... These and similar allegorical interpretations may be suitably put upon Paradise without giving offence to any one, while yet we believe the strict truth of the history, confirmed by its circumstantial narrative of facts.
On Genesis: The Refutation of the Manichees 2.3,
So then, this whole text must first be discussed in terms of history, and then in terms of prophecy. In terms of history deeds and events are being related, in terms of prophecy future events are being foretold. One should not look with a jaundiced eye, to be sure, on anyone who wants to take everything that is said here absolutely literally, and who can avoid blasphemy in doing so, and present everything as in accordance with Catholic faith; on the contrary one should hold up such a person as an outstanding and holy admirable understander of the text.
5. Venerable Bede, On Genesis:
But it must be carefully observed, as each one devotes his attention to the allegorical senses, how far he may have forsaken the manifest truth of history by allegorical interpretation.
6. St. Anastasius of Sinai, Hexaemeron1.5.2, p. 19,
We do not seek, however, to annul the literal meaning. Rather, we seek the meaning that the Holy Spirit, in its great goodness and love for humanity, mystically encrypted within the literal. Toward this end we will examine the text first in its bodily or physical sense.
Hexaemeron 1.11.3, p. 41
We, however, while making the anagogical observations that have been discussed above, are not destroying the literal meaning. Come now. We clearly know that a perceptible darkness was lying upon the perceptible deep, and that a breath of God was borne upon the perceptible water …
Hexaemeron 3.2.1, p. 63
Remember me, O reader, as one who said that we are not destroying the literal meaning, but are pursuing its unwritten anagogy.
Hexaemeron 4.1.1, p. 95
I consider it reasonable, deserving full acceptance, and helpful to remind the readers throughout our exegesis of the Hexaemeron that, when we allegorically interpret the physical creations for their spiritual significance, we do not destroy the literal story about the works of God.