Some of the difficulties faced by empirical economists is lack of data, and perhaps more specifically lack of data which is able in a detailed manner to identify complete decision patterns. On the other hand, computerization of companies and governmental agencies the last few decades have made it possible to record data on a large scale both with regards to high frequency time series, but also panel and cross sectional data. As a consequence, traditional methods are not sufficiently capable to encompass the complexity of these detailed data. Despite the application of some machine learning (ML) methods to economic problems, it is noted in the literature that ML and other forms of big data handling such as data mining not fully is embraced by the field of economics compared to that of the traditional econometric methods. The purpose of this project is to combine and apply econometric and ML methods to economic problems where big data, broadly defined, is available. While big data is a relative term depending on the contemporary computational power, the methods will be applied to both small (relative) and large data sets even while the data is not defined as big data, per se. Challenges faced when doing statistical analysis is the data handling done by companies. Often, data is recorded in regular and predefined intervals. At the same time, it is stored without being used for anything else than displaying the raw data in figures and/or tables, if even that. Consequently, a job must be done with regards to facilitating and connecting the data to the implemented algorithms. This requires commitment from the companies. A long term goal of this project is to make companies better observe and acknowledge the value their data often contains, increase their knowledge with regard to what expertise to demand, and when.
Project leader: Ingrid Kristine Pettersen
Institution: CAPIA AS