A New Artificial Intelligence (AI) Benchmark Called ‘OpenCQA,’ Answers Open-Ended Questions About Charts Using Descriptive Texts

Finding and communicating key data insights using data visualization methods such as bar charts and line graphs is essential for many activities, but it can be time-consuming and labor-intensive. Data analysis and communicating key findings are two common uses for charts. Analysis of visual representations is often used to provide explanations for problems that do not have a clear yes/no answer. It takes a lot of mental and perceptual energy to answer questions like this. Therefore it can take time and effort.

To solve these issues, the Charted Question Answering (CQA) task was developed to accept a graph and a natural language question as input and give an answer as output. The CQA has been studied extensively in recent years. However, the difficulty is that most datasets only include examples where the answer is a single word or phrase.

Since few data sources with graphs and associated textual descriptions are readily available, they have still attempted to create datasets with open-ended questions and answer statements written by annotators. Therefore, the researchers used graphs from Pew Research (pewresearch.org), where experts use various graphs and summaries to generate papers on market research, public opinion and social concerns.

A total of 7724 sample datasets were generated by adjusting the number of summary words in the 9285 graph-summary pairs extracted from approximately 4000 articles on this website. A total of 7724 records were included as part of the sample. The dataset’s many charts and graphs cover a variety of topics, from politics and economics to technology and beyond.

Also Read :  Group of Information Services (NASDAQ:III) StockNews.com has upgraded it to "Strong-Buy."

Four questions can be asked in the OpenCQA, and the answer is the output text of the task.

  • To identify it, ask questions about a certain target within a series of bars.
  • Graph comparison questions are under the “compare” category.
  • One of the options is to summarize the data in graphical form, which is what the question wants you to do.
  • Undirected queries that require conclusions across the entire graph

Models used as a starting point

The new dataset was developed with reference to the seven existing models:

  • Improved performance over the standard BERT model by adding focused attention layers, abbreviated as BERTQA
  • Models such as ELECTRA’s self-supervised representation learning and GPT-2 Transformer-based text generation could anticipate the next word in a text based on the words already used.
  • Models like BART, which use a combined encoder-decoder transformer framework, have proven to achieve excellent performance on text production tasks such as summarization.
  • Among the models that propose a document-based generation task in which text generation is improved by the model with the information provided by the document are (a) T5, a unified model of an encoder-decoder transformer for converting language tasks into text-to- text; (b) VLT5, a T5-based framework that unifies Visual-Language tasks such as text generation subject to multimodal input; and (c) CODR, a model that recommends a document-based generation task.
Also Read :  Charter's (CHTR) Slow Internet Customer Growth Impacts Prospects - December 27, 2022

Challenges and Limitations

Many ethical considerations arose for researchers while collecting and recording data. They only used charts that were freely accessible and found on publicly available resources that allow download information to be disseminated for educational purposes so as not to infringe on the intellectual property of the original producers of the chart. Users are permitted to use data from the Pew Research Center as long as appropriate credit is given to the organization, or no other entity is named as the source.

Researchers have speculated that the models could be used to spread false information. Although the outputs of the current model appear natural, they contain some inaccuracies discussed in the cited study. Because of this, releasing these erroneous model results in their current form could misinform the general public.

However, due to the nature of the task, only data from Pew Research (pewresearch.org) could be used in the analysis, limiting the dataset. In the future, if more relevant data become accessible, the data set could be expanded. Researchers also ignored long-range sequence models such as the line former and the newly proposed reminiscence transformer.

Also Read :  10 Mobile Apps To Launch A Startup In 2023

Since it can only work with the automatically produced OCR data, which is usually noisy, the job configuration is also limited. To better account for data in the model, future techniques can focus on optimizing OCR extraction for this specific job.

Finally, OpenCQA is recommended as a means of providing detailed answers to free-form charting questions. At the same time, they present some cutting-edge standards and metrics. The exam results show that while the most advanced generation models can generate natural language, there is still a lot of work to do before they can consistently provide valid arguments that use both numbers and logic.

Check the Paper. All Credit For This Research Goes To The Researchers On This Project. Also, don’t forget to join our Reddit Page, Discord channeland Email newsletterwhere we share the latest AI research news, cool AI projects, and more.

Dhanshree Shenwai is a Computer Science Engineer with good experience in FinTech companies covering the domain of Finance, Cards & Payments and Banking with a keen interest in AI applications. She is passionate about exploring new technologies and advancements in today’s changing world, making everyone’s life easy.


Leave a Reply

Your email address will not be published.

Related Articles

Back to top button