Chemical speciation of heavy metals (Zn, Pb, Cu, and Cd) was studied to evaluate the contamination status and associated risks and to constrain the sources of heavy metals in relation to sedimentary organic matter (OM) sources in surface sediments of the Cross River Estuary (CRE) and nearshore areas surrounded by a degrading mangrove ecosystem (typical C3 plants). The contamination factor (CF) and geo-accumulation (Igeo) indicated that Cd and Zn were the most polluted heavy metals. High percentages of Zn (63.78%), Pb (64.48%), Cd (76.72%) and the considerable amount of Cu (48.57%) in non-residual fractions indicated that these heavy metals are bioavailable. Cd showed moderate to high ecological and bioavailability risk based on the ecological risk (Er) and risk assessment code (RAC). Significant positive correlations occurred among the heavy metals, fine-grained sediments, and sedimentary OM from terrestrial C3 sources. These correlations, together with high percentages of heavy metals in the oxidizable fraction (~33–50%), indicated that the erosive washout of OM and fine sediments ladened with heavy metals from the adjoining degraded mangrove ecosystem contributed significantly to the increased contents of heavy metals in surface sediments of the study area.