After the war Nebe joined the police and eventually became head of the criminal police agency in Prussia. By 1931 was a member of the National Socialist German Workers Party (NSDAP) and the Schutzstaffel (SS).
When Hermann Goering became minister of the interior in Prussia in 1933 he recruited Nebe to work with Rudolf Diels in the Gestapo. Nebe was later appointed as head of the Criminal Police (Kriminalpolizei).
Nebe appears to have became disillusioned with the atrocities committed during Operation Barbarossa. It was later claimed that Nebe saved thousands of Russian civilians from execution by falsifying figures of those killed.
Suspected of being involved in the July Plot Nebe went into hiding but he was betrayed by a former mistress and is reported to have been executed on 21st March, 1945. However, there are reports of him being sighted after the war in Turin (1956) and Ireland (1960).