Warhammer 40k Lore: Where to Start -- The War in Heaven
The Warhammer 40k universe doesn't start with the Space Marines, or the Horus Heresy, or even with the rise of the the Emperor. It starts, in many ways (though much is still unknown) by the Old Ones and the C'Tan. They clashed more than once, and the final fight was called The War in Heaven. It would see the fall of both the Old Ones and the doom of the C'Tan, and lay the groundwork for the Necrons to reemerge millions of years later.
In Warhammer 40k, Who Are the Old Ones?
The Old Ones in Warhammer 40k are an ancient reptilian race, living some sixty million years before the present day (the 41st Millennium). They were a very intelligent race, intrigued by the galaxy, and they devised the Webway, a method of traveling through the galaxy without encountering the perils of the Warp. (The pursuit of this Webway was one of the Emperor's ultimate goals.)
It was during this time that the Old Ones encountered the Necrontyr.
(As a side note: are the Old Ones in Warhammer 40k the same Old Ones in Warhammer Fantasy Battles and Age of Sigmar? It appears that in early versions of Warhammer Fantasy and Warhammer 40k, they were intended to be linked. But, as 40k became less and less "fantasy in space", changing the space elves into Aeldari, getting rid of the space dwarves (squats), and minimizing the roles of ogryns (ogres) and ratlings (halflings), this link between the Old Ones appears to be severed.)
Who Were the Necrontyr?
The Necrontyr were a mortal race who lived a harsh life on a barren world. They were sickened by radiation. They lived short, terrible lives and tried over and over to somehow cure their illnesses, but never could. They were, however, able to get off their planet and into the stars. One of the technologies they crafted was living metal, which they used to shield their ships during space flight.
The Necrontyr met the Old Ones and, discovering that the Old Ones were effectively immortal they pled for information about how to live longer--how to become immortal themselves. The Old Ones refused to help them and the Nectrontyr went to war against them. However, they quickly learned that they were outmatched. The Necrontyr had no knowledge of the Webway and had to travel the galaxy in slow stasis ships.
Beaten back, the Necrontyr fled. They grew disorganized and began fighting each other, eventually into a civil war--the second War of Succession that the Nectrontyr had lived through.
However, during this time Silent King Szarekh discovered the presence of the C'Tan.
Who Were the C'Tan?
The C'Tan are believed to be as old as the universe itself, vastly older than the Old Ones. They were not mortals, but beings of pure energy that were only somewhat self-aware. They orbited stars, sucking power from the suns like vampires.
It's not certain how the C'Tan and the Necrontyr met. In some accounts the discovery of the C'Tan was accidental. In other accounts, such as the Aeldari's ancient records, say that the enormous hatred the Necrontyr had for the Old Ones was sensed by the C'Tan and they drew themselves near. Either way, at the sun of the Nerontyr's original homeworld, the first C'Tan materialized in the form of Aza'gorod the Nightbringer. He used the living metal of the Necrontyr's ship to form a body (a necrodermis).
The Nightbringer began slaughtering everyone and absorbing all their pain and suffering like a vampire. But the Necrontyr pled for their lives and negotiated servitude to the C'Tan in exchange for leading the C'Tan to more powerful beings (the Old Ones.)
How Did the Necrontyr Become the Necrons?
There was one C'Tan, Mephet'ran, who wasn't as strong as the other C'Tan, but who ingratiated himself to the Necrontyr. To them he was known as The Messenger, but to the other C'Tan he would be called The Deceiver. Mephet'ran began to spread stories to the other C'Tan about how there was once, long before, a battle between the C'Tan and the Old Ones. He wanted to rekindle an ancient feud.
For his purposes he brought about a plan which he said would help both the Necrontyr and the C'Tan, but which he was doing for his own causes: he promised the Nectrontyr immortality--the one thing they had always sought--through "biotransference": putting their souls into living metal bodies.
All the Necrontyr were biotransferred, even though some did it against their will and even organized resistance to fight it (though some of the resistance was organized by The Deceiver himself, as a way to gather all the dissenters together and capture them). As the Necrontyr were biotransferred, the C'Tan vampirically sucked their life energy away.
Thus, all the Necrontyr were immortal beings of metal--the Necrons.
What Happened in the War In Heaven?
As soulless Necrons, they fought alongside the C'Tan in a war against the Old Ones. During the war, whole solar systems were destroyed and races were eradicated.
The Old Ones had previously created new races, including one powerful psychic race--the Aeldari--which they now called as an ally in the War in Heaven. As more young races joined the Old Ones, the C'Tan began to be pushed back. The C'Tan struggled also against the power of the Warp, which they couldn't understand.
However, during all of this, the C'Tan unified with other C'Tan from all over the galaxy, all of these energy beings uniting together to fight. One C'Tan, Mag'ladroth, the Void Dragon, was able to seal off the Warp using blackstone (a substance found throughout the galaxy that can manipulate the Warp).
The C'Tan, like the Old Ones, had begun creating races (or trying to). They changed the genes of some of the creatures who would one day evolve into humans; these beings became Necron Pariahs.
The C'Tan breached the Old Ones' Webway and released the horrors of the warp into it, and the Old Ones were finally defeated.
However, just in their moment of victory, the C'Tan were to fall--they were exhausted, and the Silent King Szarkh made a move to rise up. The C'Tan were destroyed, broken into C'Tan Shards which were held in prisons called Tesseract Labyrinths. There are rumors that some may have entered stasis as energy beings and hidden.
What Was the Aftermath of the War in Heaven?
The Necrons were triumphant, but now they were the weak ones who were ready to be defeated--this time by the rising Aeldari. The Silent King knew that the Necrons were immortal and the Aeldari weren't, so he knew time was on his side. He implemented the Great Sleep, in which all the Necrons were laid to sleep in Tomb Worlds to sleep for sixty million years.
The Aeldari would go on to grow and use the technology of the Webway, becoming the most powerful race in the galaxy for nearly sixty million years, until The Fall of the Eldar. in the 31st Millennium.