TY - GEN
T1 - Skoltech at TextGraphs-17 Shared Task
T2 - 17th Graph-Based Methods for Natural Language Processing, TextGraphs 2024 - co-located with the 62nd Annual Meeting of the Association of Computational Linguistics, ACL 2024
AU - Lysyuk, Maria
AU - Braslavski, Pavel
N1 - Publisher Copyright:
© 2024 Association for Computational Linguistics.
PY - 2024
Y1 - 2024
N2 - In this paper, we present our solution to the TextGraphs-17 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering (KGQA). GPT-4 alone, with chain-of-thought reasoning and a given set of answers, achieves an F1 score of 0.78. By employing subgraph size as a feature, Wikidata answer description as an additional context, and a question rephrasing technique, we further strengthen this result. These tricks help to answer questions that were not initially answered and to eliminate irrelevant, identical answers. We have managed to achieve an F1 score of 0.83 and took 2nd place, improving the score by 0.05 over the baseline. An open implementation of our method is available on GitHub.
AB - In this paper, we present our solution to the TextGraphs-17 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering (KGQA). GPT-4 alone, with chain-of-thought reasoning and a given set of answers, achieves an F1 score of 0.78. By employing subgraph size as a feature, Wikidata answer description as an additional context, and a question rephrasing technique, we further strengthen this result. These tricks help to answer questions that were not initially answered and to eliminate irrelevant, identical answers. We have managed to achieve an F1 score of 0.83 and took 2nd place, improving the score by 0.05 over the baseline. An open implementation of our method is available on GitHub.
UR - http://www.scopus.com/inward/record.url?scp=85204911377&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85204911377&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85204911377
T3 - TextGraphs at ACL 2024 - Proceedings of TextGraphs-17: Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
SP - 149
EP - 153
BT - TextGraphs at ACL 2024 - Proceedings of TextGraphs-17
A2 - Ustalov, Dmitry
A2 - Gao, Yanjun
A2 - Panchenko, Alexander
A2 - Tutubalina, Elena
A2 - Nikishina, Irina
A2 - Ramesh, Arti
A2 - Sakhovskiy, Andrey
A2 - Usbeck, Ricardo
A2 - Penn, Gerald
A2 - Valentino, Marco
PB - Association for Computational Linguistics (ACL)
Y2 - 15 August 2024
ER -