Arunas Ciuksys and Rita Butkiene, Kaunas University of Technology, Lithuania

Automatic event extraction (EE) is a crucial tool across various domains, allowing for more efficient analysis and decision-making by extracting domain-specific information from vast amounts of textual data. In the context of under-resourced languages like Lithuanian, the development of EE systems is particularly challenging due to the lack of annotated datasets. This study investigates and evaluates the event extraction capabilities of two large language models (LLMs): OpenAI's GPT and Google Gemini, using few-shot prompting. We propose novel methodologies, including a combined approach and a layered prompting approach, to improve the performance of these models in identifying two specific event types. The models were benchmarked using various performance metrics, such as accuracy, precision, recall, and F1-score, against a manually annotated gold-standard corpus. The results demonstrate that LLMs achieve satisfactory performance in extracting events in Lithuanian, though model accuracy varied depending on the prompting methodology. The findings underscore the potential of LLMs in addressing event extraction challenges for under-resourced languages, while also pointing to opportunities for improvement through enhanced prompt strategies and refined methodologies.

Event Extraction, LLMs, Few-Shot Prompting, Gemini, GPT, Layered Prompting, Combined Prompting.