Gandalf left for Isengard to speak with Saruman. Once there, however, Gandalf discovered Saruman wished to join Sauron.
Saruman explained that they could eventually control the Dark Lord for their own benefit. Saruman also proposed they find the ring to challenge Sauron.
Upon refusing to help Saruman look for the ring, Gandalf was immediately taken prisoner by his former ally, giving in without a fight, knowing that he could not defeat Saruman. At Isengard, Gandalf was held prisoner at the top of the tower Orthanc.