In a nutshell, ancient Hebrew cosmology, as found in the Old Testament, considers the world in which we live a relatively flat disk, covered by a dome. Something like a gigantic cake stand covered with one of those classic glass domes, if you will.
As you can see in the diagram included, below the disk you would find the Sheol (that is, the place of the dead, but not necessarily Hell; actually, this Sheol is a bit more like what the Greeks called Hades) and the so-called “deep waters”, the “waters underneath” or, even more dramatically, “the great deep.”
Now above the dome, in the “outside” of the dome (who’d say?) you’d find even more water. You guessed it right: those are the “upper waters” and, above them, the “high heaven” or the “heaven of heavens”, where God Himself dwells, as can be seen in the graphic.
However, as explained by the Catholic blog St. Peter’s List, who in turn quote the Catholic Encyclopedia, the idea that the sky is an enormous solid dome is not to be found exclusively in Hebrew cosmology. Actually, it wouldn’t be a mistake to say that such an idea is, to some extent, a common heritage of ancient peoples, particularly Mediterranean.
For example, both Greeks and Romans assumed that the sky was a glass dome to which the “fixed stars” (that is, celestial bodies which did not seem to move, in relation to other stars in the night sky) were attached. However, there were some discrepancies about the material this dome was allegedly made out of. Some would say it was not made out of glass but iron or bronze instead but, of course, you can tell that was indeed hard to prove. That the Hebrews had similar ideas to those of their Mediterranean neighbors, can be clearly seen in several biblical passages.
The “firmament,” on the other hand, was considered as a sort of gap (or barrier) separating the “upper waters,” which are above the heavens, and the “lower waters” of the deep. The latter would include seas and oceans.
Now the dome covering the Earth was considered to be sitting on pillars, which were thought to be the “foundations of the earth,” and the dome itself had a series of windows, hatches or doors where the rain would fall from, descending from the “upper waters.” The most famous example of the opening of these hatches is, of course, Noah’s Flood.
Last but not least, deep within the Earth you would find Sheol. Etymologically speaking, it has been commonly assumed that the word “Sheol” comes from a Hebrew root which means “being sunk in,” “being hollow” or even “being buried in.” Consequently, it was only natural to assume that “Sheol” was either a cave, or a place down beneath the earth. In the Septuagint (the first Greek translation of the Old Testament), “Sheol” was translated using the classic Greek word “Hades,” while in the Vulgate (the Latin version), it was translated as “Inferos.”
Anyhow, in this particular case, the word “hell” is used in a very general sense to refer to (at least according to the kind of eschatology that can be found in the Old Testament) the realm of the dead, both the good and the bad.
In a sense, this “Sheol” can be thought of as less like “hell” than like a “limbo,” where the righteous waited for the death and resurrection of Christ, but since this “limbo of the righteous” disappeared after the Harrowing of Hell in the New Testament, “Sheol” often refers to the hell of the damned instead.