Gene

Big data

Quantile regression

Data stream mining

Statistics

Quantile

Robustness (evolution)

Biochemistry

Data mining

Computer science

Regression analysis

Regression

Mathematics

Estimator

Econometrics

Software engineering

Chemistry

Streaming data

Robust statistics

Oracle

Machine learning

Knowledge-Based Systems

Streaming data analysis has drawn much attention, where large amounts of data arrive in streams. Because limited memory can only store a small batch of data, fast analysis without access to the historical data is necessary. Quantile regression has been widely used in many fields because of its robustness and comprehensiveness. However, in the streaming data environment, it is challenging to implement quantile regression by the conventional methods, because they are all based on the assumption that the memory can fit all the data. To fix this issue, this paper proposes a novel online renewable quantile regression strategy, in which the resulting estimator is renewed with current data and summary statistics of historical data. Thus, it is computationally efficient, and not storage-intensive. What is more, the theoretical results also confirm that the proposed estimator is asymptotically equivalent with the oracle estimator calculated using the entire data together. Numerical experiments on both synthetic and real data verify the theoretical results and illustrate the good performance of the new method.

Renewable quantile regression for streaming datasets