HIV-1 uses the microtubule network to traffic the viral capsid core toward the nucleus. Viral nuclear trafficking and infectivity require the kinesin-1 adaptor protein FEZ1. Here, we demonstrate that FEZ1 directly interacts with the HIV-1 capsid and specifically binds capsid protein (CA) hexamers. FEZ1 contains multiple acidic, poly-glutamate stretches that interact with the positively charged central pore of CA hexamers. The FEZ1-capsid interaction directly competes with nucleotides and inositol hexaphosphate (IP6) that bind at the same location. In addition, all-atom molecular dynamic (MD) simulations establish the molecular details of FEZ1-capsid interactions. Functionally, mutation of the FEZ1 capsid-interacting residues significantly reduces trafficking of HIV-1 particles toward the nucleus and early infection. These findings support a model in which the central capsid hexamer pore is a general HIV-1 cofactor-binding hub and FEZ1 serves as a unique CA hexamer pattern sensor to recognize this site and promote capsid trafficking in the cell.