The reason for the physical difference between the coasts is that North America is slowly moving west, overriding the Pacific ocean floor, with the East Coast being what's called a "passive margin" characterized by slopes more gradual than in the west, drowned valleys (Hudson and Delaware River estuaries, Chesapeake Bay, Albemarle and Pamlico Sounds, etc.). In the west, only the two estuaries we know as San Francisco Bay and Puget Sound indent the U.S. coast, though Alaska has numerous indentations and lowlands.
The thirteen original colonies are based on royal land grants, with the land they retained after ceding western lands to the U.S. government accounting for their western boundaries. The Carolina colony was split into North and South when the colonial government in Charleston proved unable to give adequate government to the Albermarle settlements. Delaware was split off from Pennsylvania for reasons I'm not clear on.
Maine resulted from the splitting off of that part of Massachusetts separated from the rest of the state by New Hampshire. Vermont was originally contested between New Hampshire and New York, and declared its separate independence, joining the union just after Rhode Island. West Virginia resulted from the separate admission to the union of the Virginia counties which remained loyal to the Union at the time of the Civil War. Kentucky and Tennessee were formed by lands claimed by Virginia and North Carolina west of the Alleghenies, respectively, which were ceded to the Federal government in the 1780s. A similar process gave rise to Alabama and Mississippi, being comprised mostly of land ceded by Georgia, with a strip at the north from South Carolina's claim and their Gulf Coast being detached from the Florida Panhandle. Florida was a former Spanish colony bought by the U.S., less the pieces detached as noted above. Louisiana was made up of the populated part of the Louisiana Purchase, its east and west boundaries being from the purchase and its straight north boundary segmenting it off from the unsettled lands north. Ohio, Indiana, Wisconsin, Michigan, Illinois, and part of Minnesota were created from the Northwest Territory, land where Virginia, Pennsylvania, and several other states had claims which they ceded to the Federal government. (Northeastern Ohio was originally the Western Reserve of Connecticut, land that that state retained when giving up its other claims.) Most of the other states have boundaries resulting from either (a) straight lines on the map drawn to demarcate territories prior to statehood, (b) rivers adopted as boundaries, or (c) the "divide" line connecting mountain summits separating watersheds.