Team Name : Team Chaos
Problem Description : A software which will analyse and highlight the important ('tricky') clauses in an End-User License Agreement (EULA).
Abstract of the Idea : When you install a new application, you are prompted to accept the Terms & Conditions or the EULA before you can start using the application. Most of us ignore these EULAs due to lack of legal knowledge & the length of the documents. But, these EULAs are legally binding & their violation can lead to severe legal implications.
Scraped Dataset consists of 23K EULAs with around 400K Datapoints. Due to lack of legal expertise and lack of automated methods to analyse the clauses, we gathered 3K Datapoints, rated them and used them for training.
We trained 2 Natural Language Models -
-
Word & Document Vector Similarity-based Model We used Gensim's Doc2Vec & Word2Vec models to convert the legal clauses into vectors & then trained a simple binary classifier on those vectors. We observed the best results for NUM_DIM=300. A simple Cosine similarity-based classifier also yeilded similar results. We were able to achieve an accuracy of 72.7%. But, the Deep Learning based approach outperforms this Similarity-scoring approach by a huge margin.
-
BiDirectional Multi-Layer LSTM Model We implemented a Bi-directional multi-layer LSTM Model in Pytorch and trained it on our dataset. We achieved best results for NUM_DIM=400, NUM_LAYERS=2. We achieved an accuracy of 96.0%. Inference was carried out on never-before-seen data from EULAs. The model can correctly guess tricky clauses from unseen data.
The application is deployed on a Flask server that can be run locally with the following commands -
- Create a Virtual Environment & Install all requirements
python3 -m venv env1
source env1/bin/activate
pip install -r requirements.txt
- Run the Flask Server
python3 flaskApp.py