Human-in-the-loop soft prompt tuning for adverse drug event extraction from clinical notes

Adverse drug events (ADEs) are significant healthcare issues, increasing costs and medication durations. Extracting ADE information is crucial for enhancing healthcare delivery and drug safety. Transformer-based models' performance has recently improved through soft prompt tuning in the ADE task. Current ADE extraction models require large, annotated datasets and offer limited interpretability for clinical use. In addition, the complexity and black-box nature of transformer-based models limiting incorporating human preference and make interpretation difficult, which are critical in healthcare, hindering its full adoption. To mitigate this problem, this research proposes a human-in-the-loop learning method to tune transformers with domain expert input to complement limited data and incorporate expert preference. The method evolves in two phases: initially soft prompt tuning the model for multi-task learning of dual sequence labelling, ADE extraction with an additional soft prompt that guides the model, followed by iterative prediction, validation, and feedback refinement to optimise the model. Visualisation techniques display model predictions to users, enhancing understanding. The self-attention weights aid in diagnosing and explaining the model using saliency maps and attention flow diagrams. A graphical user interface allows experts to provide corrective labels for misclassified samples, thus refining the model. The result of the final model evaluated on testing sets from TAC 2017 and N2C2 2018 datasets, achieving state-of-the-art performance of 0.9404 and 0.9132 for N2C2 concept and relation extraction and 0.8723 and 0.5506 for TAC 2017 concept and relation extraction. In conclusion, this research demonstrates the effectiveness of incorporating human feedback to improve model performance in complex scenarios, demonstrating its effectiveness in improving drug safety surveillance and drug monitoring.

Actions (login required)

View Item