Arch, I can't post in the Judaism DIR, but the early Midrashists believed it was in fact fallen angels and giants, this is evidenced with the Dead Sea Scrolls, the Book of Enoch, and the so-called Pseudipigrapha. The idea that it wasn't Fallen angels came much later, and was originally a Sethian Gnostic idea, but originally the concensus was that they were in fact fallen angels.
