The recent development of immunotherapy as a cancer treatment has proved effective over recent years, but the precise dynamics between the tumor microenvironment (TME), nontumor microenvironment (NTME), and the systemic immune system remain elusive. Here, we interrogated these compartments in hepatocellular carcinoma (HCC) using high-dimensional proteomic and transcriptomic analyses. By time-of-flight mass cytometry, we found that the TME was enriched in regulatory T cells (Tregs), tissue resident memory CD8<sup>+</sup> T cells (T<sub>RM</sub>s), resident natural killer cells (NK<sub>R</sub>s), and tumor-associated macrophages (TAMs). This finding was also validated with immunofluorescence staining on Foxp3<sup>+</sup>CD4<sup>+</sup> and PD-1<sup>+</sup>CD8<sup>+</sup> T cells. Interestingly, Tregs and T<sub>RM</sub>s isolated from the TME expressed multiple markers for T-cell exhaustion, including PD-1, Lag-3, and Tim-3 compared with Tregs and T<sub>RM</sub>s isolated from the NTME. We found PD-1<sup>+</sup> T<sub>RM</sub>s were the predominant T-cell subset responsive to anti-PD-1 treatment and significantly reduced in number with increasing HCC tumor progression. Furthermore, T-bet was identified as a key transcription factor, negatively correlated with PD-1 expression on memory CD8<sup>+</sup> T cells, and the PD-1:T-bet ratio increased upon exposure to tumor antigens. Finally, transcriptomic analysis of tumor and adjacent nontumor tissues identified a chemotactic gradient for recruitment of TAMs and NK<sub>R</sub>s via CXCR3/CXCL10 and CCR6/CCL20 pathways, respectively. Taken together, these data confirm the existence of an immunosuppressive gradient across the TME, NTME, and peripheral blood in primary HCC that manipulates the activation status of tumor-infiltrating leukocytes and renders them immunocompromised against tumor cells. By understanding the immunologic composition of this gradient, more effective immunotherapeutics for HCC may be designed.