Big data

Algorithm

Ecology

Data envelopment analysis

Biology

Linear programming

Curse of dimensionality

Convergence (economics)

Operating system

Economics

Scalability

Rule of thumb

Data mining

Computer science

Mathematical optimization

Decomposition

Multithreading

Operations research

Database

Mathematics

Decision maker

Geometry

Point (geometry)

Economic growth

Machine learning

Thread (computing)

Computers & Operations Research

In general, data sets are growing larger and larger, and handling related issues is topic of big data. Similar trends and tendencies are evident in data envelopment analysis (DEA). DEA is a well-known instrument for determining the efficiencies of decision-making units (DMUs), applying linear programming. Still, as we will show, DEA suffers notably from the curse of dimensionality. Therefore, we propose improved decomposition-based algorithms involving different termination criteria and multithreading to address this issue. For some of these criteria, we prove the convergence of the algorithm; to the best of our knowledge, we are the first to prove this. Ultimately, from a computational point of view, we study the performance of the new big data strategy by an extensive numerical analysis, thus demonstrating the algorithm's scalability.

Big data efficiency analysis: Improved algorithms for data envelopment analysis involving large datasets