ADP-ribosylation factors (ARF) GTPases are activated by guanine nucleotide exchange factors (GEFs) to support cellular homeostasis. Key to understanding spatio-temporal regulation of ARF signaling is the mechanism of GEF recruitment to membranes. Small GEFs are recruited through phosphoinositide (PIP) binding by a pleckstrin homology (PH) domain downstream from the catalytic Sec7 domain (Sec7d). The large GEFs lack PH domains, and their recruitment mechanisms are poorly understood. We probed Golgi recruitment of GBF1, a GEF catalyzing ARF activation required for Golgi homeostasis. We show that the homology downstream of Sec7d-1 (HDS1) regulates Golgi recruitment of GBF1. We document that GBF1 binds phosphoinositides, preferentially PI3P, PI4P and PI(4,5)P2, and that lipid binding requires the HDS1 domain. Mutations within HDS1 that reduce GBF1 binding to specific PIPs in vitro inhibit GBF1 targeting to Golgi membranes in cells. Our data imply that HDS1 and PH domains are functionally analogous in that each uses lipid-based membrane information to regulate GEF recruitment. Lipid-based recruitment of GBF1 extends the paradigm of lipid regulation to small and large GEFs and suggests that lipid-based mechanisms evolved early during GEF diversification.