Recently Leutheusser and Liu [1,2] identified an emergent algebra of Type III$_1$ in the operator algebra of ${\mathcal N}=4$ super Yang-Mills theory for large $N$. Here we describe some $1/N$ corrections to this picture and show that the emergent Type III$_1$ algebra becomes an algebra of Type II$_\infty$. The Type II$_\infty$ algebra is the crossed product of the Type III$_1$ algebra by its modular automorphism group. In the context of the emergent Type II$_\infty$ algebra, the entropy of a black hole state is well-defined up to an additive constant, independent of the state. This is somewhat analogous to entropy in classical physics.