I checked Event Viewer instead of Task Manager and finally saw what was spiking my CPU
One WMI failure loop explained hours of random high usage.
This is a summary aggregated from MakeUseOf. Read the complete article on the original site:
Read full article at MakeUseOf