Physics

Pure mathematics

Mathematics

Operating system

Engineering

Usability

Power (physics)

Geography

Parallels

Data science

Scale (ratio)

Cosmos (plant)

Computer science

Art history

Cartography

Quantum mechanics

Big data

Field (mathematics)

Reliability (semiconductor)

Mechanical engineering

Proceedings of the VLDB Endowment

The twenty-first century has been dominated by the need for large scale data processing, marking the birth of big data platforms such as Cosmos. This paper describes the evolution of the exabyte-scale Cosmos big data platform at Microsoft; our journey right from scale and reliability all the way to efficiency and usability, and our next steps towards improving security, compliance, and support for heterogeneous analytics scenarios. We discuss how the evolution of Cosmos parallels the evolution of the big data field, and how the changes in the Cosmos workloads over time parallel the changing requirements of users across industry.

The cosmos big data platform at Microsoft