I agree. Not only does Scripture not mention that Joseph had children from a previous marriage, it doesn't even mention such a previous marriage. The only thing I was unsure about in your post was the statement that Jesus was almost 40 when He was crucified. I thought that He was 30 when He began His public ministry, which lasted about 3 years, so He would have been 33.
Jesus birth:
The age of Jesus comes from part historical information of what year Heard the Great died (He died in February of 4 BC.) and the age of Jesus when the Magi came. (Jesus was approximately two years old.) So this sets the birth of Christ back to at least 6 BC. And given that it took the Magi two years to get to Jerusalem; the most probable year of Jesus's birth would have been 7 BC. (Probably during the feast of tabernacles in the fall; for "I shall tabernacle among you". = I.E. the incarnation.)
We know John the Baptist was 6 months older than Jesus. Reconstructed Herod's temple was "up and functioning" in 9 BC. The first Passover of the newly constructed temple would have been in 8 BC. John's father was a priest "in the course of Abia" (Luke 1:8) which would have been the spring; (probably passover considering there were a lot of people present). This was also the first time God had spoken since the completion of the Old Testament; some 400 years(ish) earlier. John's father had his angel visit during his priest duties. And those duties would have ended by summer time. He goes back home; Elizabeth conceives John. (late summer early fall). Which John would have been born close to Passover of the following year and Jesus would have been born in the fall. Which would have been 7 BC.
It seems from Scripture, that the Magi were probably more than one group who converged on Jerusalem at the same time. It had been two years since they "saw the star in the east"; (Matthew 2:9) yet another passage says they "came from the east". (Matthew 2:1) So one group "sees the star in the east" (which means they would have been traveling from the west of Bethlehem) and the other says they "came from the east". Thus my hypothesis is that there was Magi coming from Africa, Magi coming from Asia (India or even closer to China) and possibly even Magi coming from Europe. And these were caravans of people. (Not just 3 guys on a camel)

And they would have been traveling with wives and kids and servants and soldiers etc. Matthew 2:3 says that Herod is troubled and all of Jerusalem with him. So these caravans that showed up in Jerusalem; the whole city would have known.
Thus the birth of Christ was a big deal.
Jesus death:
Now fast forward to the end of Jesus's life. Pinpointing the year he died. We got several things; both of secular records and what possible years Passover fell from a Thursday into Friday.
Secular records: Here's where it gets interesting! John the Baptist appears in the wilderness at the commencement of the 15th year of Tiberius's reign. (Luke 3:1) That would have been the summer of 29 AD. Now two very interesting things happened in the summer of 29 AD. and the Romans record these events.
One was a total solar eclipse that involved the entire Meditation area. It was witnessed in Rome, as well as in Judea. The Romans looked at eclipses as omens of the reign of the emperor. Now they may have been good omens or bad omens and since Tiberius had been a successful and popular emperor; they saw this eclipse as the "gods" looking favorably on Tiberius's reign. The Romans believed that if they did not offend anyones god, that helped to preserve the power of the empire. So they saw this as a positive omen.
Secondly, there was an earthquake in Turkey that affected Judea. This happened at the same time as the eclipse! (It was a big earthquake; like estimated 7 to 8 on the Richtor Scale.) It damaged the temple complex and caused one of the doors in the temple to get stuck open. The temple had two large doors and beyond them was the menorah, the table of shew bread and then the holy of holies. The holy of holies was covered by curtains. Called "the veil". (Keep that in the back of your mind because it's important for being able to figure out where Golgotha was.)
This earthquake also damaged the judgement hall of the Sanhedrin; thus where the trial of Jesus should have been held, wasn't held there. It ended up being held at the house of the high priest instead of the judgement hall of the Sanhedrin; because it was deemed too damaged to be used. The judgement hall was to be next to the temple; so when they had to decide any important matter; God would be present. (There are instructions to this in Leviticus or Deuteronomy; but I'm not sure where at the moment.)
So the judgement hall of the Sanhedrin was damaged; and there was a walkway above the wall, somewhere in that area that also was damaged; and this walkway was attached to the tower of Siloam which collapsed at some point and killed a bunch of people. (This is recorded in Luke 13:1-5.) This probably happened the year before Jesus was crucified. It also says Pilate sent soldiers into the temple and killed several Galileans "who's blood Pilate had mixed with their sacrifice".
Now what had happened that Pilate had sent soldiers into the temple? That wasn't a common thing. The Romans didn't generally interfere with the religious practices of the people in the empire; unless there was a believed justification for doing so. And the most probable reason Pilate did this, was because these Galileans were engaging in a ritual human sacrifice. Human sacrifice had been outlawed in the Roman empire about 90 BC.
Now who are these Galileans and why are they performing this human sacrifice? The most likely answer to that comes out of Greek history. This goes back to Maccabees. According to the Jewish record of the Maccabean revolt; this was because the Greek king was prohibiting animal sacrifices and had shut down the temple. (This is true; this did happen. For about 15 years the Seclud empire had outlawed sacrifices in the Jewish temple.) But the reason that King Antiochus IV had done so (according to Greek historians) was because he'd come across a captive Greek merchant in the temple; who'd been told by his captors, that he was to be human sacrifice of a vow the Jews had made to "hate the Greeks forever". This was the legend (Jews participating in human sacrifices. Which the Bible itself does say they did.) past through Greek historians. Although the archeological records that have been found from that era, simply say the army of the Seclud empire intervened in civil unrest that was a conflict between different sects of Judaism. This was the time of the emerging sects between Sadducees and Pharisees. So, the truth is probably somewhere in the middle of all these legends and accounts.
Now I give you this info because it becomes important when we get to Pilate and Jesus's crucifixion. Pilate was "on probation" from Rome for a revolt he put down happening in Samaria a year or so before the crucifixion. That revolt may have been connected to this incident in the temple, where one of the rioters could have been Barabbas, who was demanded to be released of the people instead of Jesus. So, background information of the conflict between the Jewish leadership and Pilate.
Later on (35 or 36 AD) Pilate was called back to Rome. Roman records don't state specifically why; but civil unrest in Judea was escalating and this recall may have had something to do with the crucifixion of Jesus and the commencement of the spread of Christianity through the empire. Pilate disappears from the record of secular history at this point. He is never recorded to have stood before Tiberius because Tiberius died in 37 AD. Some legends say Pilate committed suicide; but the eastern church states that Pilate was banished to Patmos and had become a Christian.
So the earthquake and the eclipse happened in summer of 29 AD. We know this for sure because the Romans minted a coin that came out in early 30 AD. On the front of this coin had Tiberius with the eclipsing sun behind him and on the back of the coin, was an image of the temple in Jerusalem.
Now, the book of Luke tells us that in the commencement of the 15th year of Tiberius Caesar, John appears in the wilderness. (Luke 3:1) We know that's summer / fall of 29 AD. The Feast of Trumpets starts usually in September or October; which is the start of the Jewish secular year.
I have come to the conclusion that at least part of the prophetic fulfillment of the Feast of Trumpets is connected to Jesus's baptism. This makes the most sense in the context of the leaders coming to see John out in the Jordan river compelled by these cosmic events (eclipse and earthquake). They all knew back when Herod the Great died that these caravans of people had come looking for "the king of Isreal". Herod tried to kill Jesus. They'd fled to Egypt. Some 12 years later this kid shows up in the temple; whom his parents had to back track and go get! So they knew the Messiah had been born and things were happening that were fulfillment of Scripture. Thus the interest to go see John baptizing in the wilderness.
A lot of people conclude that the Feast of Trumpets is the end of time. There may be some truth to this also; as "the last trump" stated in Corinthians is the final resurrection of the dead. So, I don't know. There could be more than one fulfillment?
So this puts the beginning of Jesus's ministry in 30 AD.
3.5 (ish) years later; Passover of 33 AD Jesus is crucified.
Thus If he was born in the fall of 7 BC and died in the spring of 33 AD he would have been 6 months shy of turning 40 years old.
Now the verse in Luke 3:23 that says Jesus was "about 30 years old". A more accurate rendering of this would be "about the middle of 30's". It's a weird numeric rendering which means "half of" or "middle of" a given number. And we know obviously Jesus wasn't 15 when he started preaching. So the "middle" or "half of" would be more properly translated "middle 30's". Which makes sense he would have been about 36 years old.
Now, the days of the week? We need a year the crucifixion would have happened during Pilate's governorship that puts Passover on a Thursday at sundown into Friday at sundown. And the only year that fits that criteria is 33 AD. Thus we can pretty well pin down that's the year Jesus died.
Pilate tried really hard to NOT put Jesus to death, and the political implications for Pilate probably played a role in this. I do believe Pilate genuinely understood Jesus to be innocent of any reason Rome would have to execute him. The Jewish leadership's first complaint was that Jesus had committed blasphemy by stating he's the Son of God. (They full well knew he was their Messiah.) But it wasn't a crime in the Roman empire to claim to be God. And this is why Pilate told them Jesus had done nothing worthy of death. Also though, (besides the dreams of Pilate's wife) I'm sure Pilate had heard the "rumors" that this guy could heal the sick and raise the dead. And Scripture says that Pilate knew that they'd brought Jesus to him out of their envy. (Matthew 27:18, Mark 15:10)
Now.... finally in conclusion:
Back to the earthquake and the door of the temple that got stuck open; which clues us to the location of Golgotha. Scripture tells us that the soldiers at the site of the crucifixion actually witnessed the veil of the temple being torn; (which was followed by another earthquake)! (Matthew 27:51-54; also recorded in Mark 15:39) Thus in order for them to actually see the veil in the temple being torn; they would have had to be facing the temple, in a location where they could see through the door that was stuck open. Which means Golgotha was on the Mt. of Olives probably somewhere in the vicinity of the Red Heifer Alter.
There's several really interesting passages that indicate that the Roman soldiers who crucified Jesus, knew more about Jewish prophecy than we would suspect they would. Keeping in mind this legion had been "stationed" in this area for a decade before Jesus was crucified. And the soldiers from this legion would have been present in the area of the temple when Jesus performed these various healings and other miracles. We even have records in Scripture of centurions and soldiers coming to Jesus asking instructions and for servants and family members to be healed.
Now, last little tidbit of information I'll throw out here. "3 days and 3 nights in the heart of the earth":
It's becoming (somewhat) popular for people to claim Jesus was crucified on Wednesday (to get to 3 days and 3 nights - assuming "heart of the earth" means "in the grave". Others say "one part of a day counted as a day". Seeing how clearly Jesus was not in the tomb 3 full days and 3 full nights. (i.e. he didn't rise from the dead on Tuesday!) He did rise on Sunday and the Greek in the New Testament regarding resurrection day and day of church gathering, is called (a) "Sabbath". (But that's a whole other rabbit hole to go down; and this response is long enough as it is.)
The "3 days and 3 night in the heart of the earth" though is the time span between Jesus last public ministry event when he left the temple Tuesday evening and his death on Friday before sundown. That's exactly 3 (three) 24 hour periods and that span of time entails when the atonement took place.
At the point Jesus died; the payment for sin was complete. He says "it is finished" and he dies and ascends to heaven. We pick this up in Revelation 5 when he (as a disembodied soul) is standing before the throne of the Father as "the lamb slain" to open the seals. Sunday morning (probably about 3 AM) his soul (and 2nd person of the Trinity divine nature) descends back to earth, is reunited with his flesh and rises from the dead. For the last enemy to be destroyed is death. (1 Corinthians 15:26) That is the final indicator to the rest of us that one day this material cosmos will also be raised from corruption. "For if Christ be not raised; you are yet in your sins." (1 Corinthians 15:17)
Now aint that wild!