Bismuth is a lithium-ion battery anode material that can operate at an equilibrium potential higher than graphite and provide a capacity twice as high as that of Li4Ti5O12, making it intrinsically free from lithium plating that may cause catastrophic battery failure. However, the potential of bismuth is hampered by its inferior cyclability (limited to tens of cycles). Here, we propose an "ion conductive solid-state matrix" approach to address this issue. By homogeneously confining bismuth nanoparticles in a solid-state γ-Li3PO4 matrix that is electrochemically formed in situ, the resulting composite anode exhibits a reversible capacity of 280 mA hours per gram (mA h/g) at a rate of 100 mA/g and a record cyclability among bismuth-based anodes up to 500 cycles with a capacity decay rate of merely 0.071% per cycle. We further show that full-cell batteries fabricated from this composite anode and commercial LiFePO4 cathode deliver a stable cell voltage of ∼2.5 V and remarkable energy efficiency up to 86.3%, on par with practical batteries (80-90%). This work paves a way for harnessing bismuth-based battery chemistry for the design of high capacity, safer lithium-ion batteries to meet demanding applications such as electric vehicles.