The evidence for Genesis 2v4 onwards being a different record to Gen1 is based on more than the contrasts between the two chapters (though that ought to be enough). Further evidence is the expression “these are the generations of…” in Gen2v4 – the formula repeated 11 times in Genesis to introduce a new historical narrative.
The following are the notes from the NET on the phrase:
“The Hebrew phrase “elle toledot” is traditionally translated as “these are the generations of” because the noun was derived from the verb “beget.” Its usage, however, shows that it introduces more than genealogies; it begins a narrative that traces what became of the entity or individual mentioned in the heading. In fact, a good paraphrase of this heading would be: “This is what became of the heavens and the earth,” for what follows is not another account of creation but a tracing of events from creation through the fall and judgment (the section extends from 2:4 through 4:26). See M. H. Woudstra, “The Toledot of the Book of Genesis and Their Redemptive-Historical Significance,” CTJ 5 (1970): 184-89.
The expression this is the account of is an important title used throughout the Book of Genesis, serving as the organizing principle of the work. It is always a heading, introducing the subject matter that is to come. From the starting point of the title, the narrative traces the genealogy or the records or the particulars involved. Although some would make the heading in 2:4 a summary of creation ( 1:1-2:3), that goes against the usage in the book. As a heading it introduces the theme of the next section, the particulars about this creation that God made. Genesis 2 is not a simple parallel account of creation; rather, beginning with the account of the creation of man and women, the narrative tells what became of that creation. As a beginning, the construction of 2:4-7 forms a fine parallel to the construction of 1:1-3. The subject matter of each “toledot” – “this is the account of” section of the book traces a decline or a deterioration through to the next beginning point, and each is thereby a microcosm of the book which begins with divine blessing in the garden, and ends with a coffin in Egypt. So, what became of the creation? Ge 2:4-4:26 will explain that sin entered the world and all but destroyed God’s perfect creation.”
Note the AMG Word Study Dictionary and TWOT both support the NET notes on the Hebrew here indicating a new section and new piece of history as the Hebrew does throughout Genesis.
Similarly Walton concludes
there is therefore no precedent by which to conclude that the introductory formula in Genesis 2:4 is bringing the reader back into the middle of the previous account to give a more detailed description of a part of the story that was previously told. Such introductions never do this in the rest of Genesis, and the word tōlĕdōt (account) argues against such an understanding. Furthermore, Genesis 2 does not follow the pattern of the recursive examples that follow a genealogy of the unfavored line before returning to the story of the favored line. This evidence then leads us to give strong preference to the view that Genesis 2 is not adding further detail to what happened during the sixth day in Genesis 1. It would therefore also mean that, though Adam and Eve may well be included among the people created in Genesis 1, to think of them as the first couple or the only people in their time is not the only textual option.
(Walton, J. H. (2015). The Lost World of Adam and Eve: Genesis 2–3 and the Human Origins Debate (p. 66). Downers Grove, IL: IVP Academic: An Imprint of InterVarsity Press.)