Duality Cache is an in-cache computation architecture that enables general purpose data parallel applications to run on caches. This paper presents a holistic approach of building Duality Cache system stack with techniques of performing in- cache floating point arithmetic and transcendental functions, enabling a data-parallel execution model, designing a compiler that accepts existing CUDA programs, and providing flexibility in adopting for various workload characteristics. Exposure to massive parallelism that exists in the Duality Cache architecture improves performance of GPU benchmarks by 3.6x and OpenACC benchmarks by 4.0x over a server class GPU. Re-purposing existing caches provides 72.6x better performance for CPUs with only 3.5% of area cost. Duality Cache reduces energy by 5.8x over GPUs and 21x over CPUs.