Using large language models (LLM) like GPT-4 in data analysis costs less than 1 per cent of hiring a human analyst while turning in comparable performances, according to a recent study that highlights the potential threat to job security amid increased adoption of generative artificial intelligence (AI) in various industries.
The cost of GPT-4, the latest version of the LLM developed by US start-up OpenAI, is only 0.45 per cent of hiring a senior data analyst who earns a market rate of around US$90,000 annually, or 0.71 per cent of a junior-level employee, according to findings by researchers from Damo Academy, the in-house research arm of Chinese e-commerce giant Alibaba Group Holding, and Singapore’s Nanyang Technological University, posted on the preprint server arXiv.
Alibaba owns the South China Morning Post.
In the study, the researchers gave questions and provided the data, and automated the whole data analytics process using GPT-4 to extract and analyse data, to eventually produce insights and graphs.
The results of the analysis were compared with professional human data analysts in terms of performance, time and costs.
The experiments showed that GPT-4 is not just significantly cheaper than a human data analyst, but also much faster in completing the tasks, according to the study findings.
However, the study noted that further studies are needed before concluding that GPT-4 can replace data analysts.
GPT-4 can also beat an entry-level human analyst in terms of performance, which was evaluated through a range of metrics including the correctness and fluency in charts and the insights they produced, and has “comparable performance” to a senior level analyst, with advantages varying among different cases and metrics.
In some cases, the AI model managed to surpass the human data analysts in terms of the correctness of the figures and analysis, and the insights GPT-4 generated tended to be more complex, according to the study.
GPT-4 also got full marks in alignment and fluency in generating the analysis with grammatically correct texts.
However, GPT-4 fell behind humans in terms of showing correct data in graphs, as well as presentation and formatting in some cases.
Despite errors with some figures, GPT-4 could still generate correct analysis, the study pointed out.
The study sheds new light on the expanding application of LLMs in various industries, where it is expected to improve efficiency but also threaten human jobs.
Chinese tech companies are racing to develop their own ChatGPT-like services.
Alibaba is working on its own answer called Tongyi Qianwen, which it launched in April for beta testing to corporate clients by invitation, weeks after Baidu unveiled its Ernie Bot service. South China Morning Post