Overview:

The Distribution Economics Institute of Japan (DEIJ) is known for its pioneering research and survey activities in the distribution and marketing field. DEIJ constructed a tool called ‘Marketer’s Desk’ that rapidly searches, aggregates, and analyzes big data collected and stored from a variety of public organizations and private companies such as drug stores, convenience stores, and supermarkets using RDB and statistical analysis package. The tool performs multilateral survey and analysis to understand consumers behavioral mechanisms. DEIJ quickly adapts to environmental changes concerning distribution, and aims to help improve as well as innovate marketing and merchandising during manufacturing and trading. The data search and analysis tool however has recently been renovated by collaborating with Unicage as well as NTT Data.

Challenge:

Due to the replacement of PC servers, the tool needed an update. Additionally, as the data collected over the years grew, the time it took to analyze the data also grew proportionally. A scalable alternative for the program was therefore necessary to account for the ever-growing data. In fact, the data accumulated daily has grown to a record 2 billion in just the last 4 years. In order to respond to the demands for an advanced analysis tool, data analysis was performed using foreign statistical tools. However, due to the over customization of these tools, it became non-augmentable and certainly inflexible.

Solution:

The Unicage system was implemented as a sort of query language that aids the rapid search of complex data. Using Unicage, DEIJ also managed to transfer the original Marketer’s Desk’s analysis function as well as formulas used in aggregation index and conditional branch’s logic into the new system. Unicage supplemented the program by ensuring scalability even as the data being processed expanded proportionally. DEIJ developed a total of 9 types of analysis menus based on 10 types of data sources. The ID-POS analysis menu included purchase attribute analysis and basket analysis. Furthermore, basic analysis menus such as sales ABC analysis, sales trend analysis, selling price analysis, and purchase switch analysis were developed. In addition to the existing analysis functions, new features were also added such as aggregation and drill down analysis, thus illuminating the flexible scalability of Unicage, as it is able to grow and adapt to contextual specificities.

Tamae Takakuwa:DEIJ System administrator

“Because managers can run simple UNICAGE commands themselves now, their task burden is minimized and researchers could rapidly obtain reports.”

Despite the data volume increasing 1.5 times to 450GB, search speed has improved even with the same hardware environment

Improved productivity in the workplace as employees can now rely on simple Unicage commands to directly process data themselves instead of relying on managers who previously used complex SQL queries to extract, process, and report data

Maintainability and scalability of the software even with growing data