Detect AI Generated Content with Large Languge Models

Introduction

The escalating use of Generative AI in content creation has given rise to the widespread dissemination of AI-generated content across blogs, articles, and newsletters. Some people might leverage it as a side income source, particularly when they incorporate Google Ads into their blogs featuring autogenerated content. However, the content generated using AI could mislead viewers because the inherent nature of Generative AIs are just predictive models; they lack ethical considerations. They might portray harmful and false information based on their dataset, and even detailed prompts failed to mitigate the bias and existing discrimination. To address this issue, I propose a project that leveraging my knowledge to categorise content, label them as either AI-generated or Human-written. With that, it will let reader to exercise caution when they are reading and analysing the content, bearing in mind that AI-labeled content is solely based on mathematical model.

My Plan

In this project, I am planning to do a Literature Review in the context of Large Langugae Model’s progression; identifying the types, advantages, disadvantages of models, and how throughout the years the more “advance” models are replacing the “old” ones. Then, I will generate the dataset for this project using different service provider’s APIs. Finally using National Supercomputing Center’s resources to run my LLM fine-tuning, and result comparison. Apart from the usual flow, I also would like to develop hybrid model and see if this approach will generate a better performance.

When research for similar work on classifying AI-generated content, I stumbled across a paper named - Arabic ChatGPT Tweets Classification Using RoBERTa and BERT Ensemble Model. One interesting approach is they use Hybrid transformer-based model. Developed by combining the hidden outputs of the RoBERTa and BERT models using a concatenation layer, then adding dense layers with Relu activation employed as a hidden layer to create non-linearity and a softmax activation function for multiclass classification.

I find their approach to be greatly inspirational! And if you’re ready, let’s explore together!

Project Links

Detect AI Content with LLM Part 1: Literature Review

Literature Review - Large Language Model (LLM) In the context of classifying AI-generated content, Large Language Models (LLMs) are commonly used because it can capture the nuances in the language and can effectively achieve general-purpose language understanding. In practice, LLMs are trained on massive unlabeled dataset before being deployed for specific downstream tasks, with little changes in the architecture post-training. Training for a Large Language Model requires high computational power, usually trained distributedly in multiple GPUs / TPUs for days / months....

Detect AI Content with LLM Part 2: Data Generation and Pre-processing

Dataset Used For this project, I utilized data from Kaggle which comprises of test_essays.csv, train_essays.csv and train_prompts.csv. {test|train}_essays.csv consists of four columns: id (unique identifier), prompt_id (identified the prompt that was used to generate the essay), text (the essay) and generated (1 if generated by LLM, 0 if generated by human). train_prompts.csv includes prompt_id (unique identifier), prompt_name (title of the prompt), instructions (instruction given to students) and source_text (the text of the article the essays were written in response to)....

Detect AI Content with LLM Part 3: Compute Engine and Result Comparisons

Compute Engine With data generation and pre-processing completed, it is time to fine tune our model with the collected data. So before I actually do the training, I thought I can finely use Google Colab Free Tier Account to do all the computation, since TPU is available to us. Hoever, my assumption is proved to be incorrect. I tried to use Google Colab TPU for the XLNET fine-tuning process, the process took more than half a day (until it finally returned error because my browser disconnected from the internet, how frustrating it is!...

Introduction#

My Plan#

Project Links#

Introduction

My Plan

Project Links