Transferring entangled states between photon pairs is essential for quantum communication technologies. Semiconductor quantum dots are the most promising candidate for generating polarization-entangled photons deterministically. Recent improvements in photonic quality and brightness now make them suited for complex quantum optical purposes in practical devices. Here we demonstrate for the first time swapping of entangled states between two pairs of photons emitted by a single quantum dot. A joint Bell measurement heralds the successful generation of the Bell state $\Psi^+$ with a fidelity of up to $0.81 \pm 0.04$. The state's nonlocal nature is confirmed by violating the CHSH-Bell inequality. Our photon source is compatible with atom-based quantum memories, enabling implementation of hybrid quantum repeaters. This experiment thus is a major step forward for semiconductor based quantum communication technologies.