AI Detector

Training a robust binary classifier on AI-generated vs. human-written text requires a large, balanced dataset small datasets overfit to specific generators or writing styles. Extracting 29 NLP features across 300,000 texts is computationally expensive and can take hours on a single GPU. If the process crashes at row 150,000, restarting from zero wastes half a day of GPU time.

Assembled a ~299,000-row dataset (150,000 AI-generated, 149,128 human-written) split across three CSV files. Built a resumable feature extraction pipeline in train.py: it writes features to a CSV incrementally (flushing after every 100 rows), reads back the existing output file on restart to determine the start index, and resumes extraction from where it left off. The 29 features are used to train the model on a 95/5 stratified split with a fixed random seed (42) for reproducibility.

A resumable pipeline was necessary because the full extraction run takes 6-8 hours on a Kaggle GPU. Kaggle kernels have a 12-hour time limit, and connectivity interruptions are common. The incremental-write design means every 100 rows are persisted to disk, so the maximum wasted work on a crash is 100 rows, not 150,000. The robust_load_csv function handles malformed rows (quoted text containing commas, empty records, encoding errors) with a regex-based parser that finds record boundaries by matching the trailing label character ([01]) at the end of each CSV line, rather than relying on a naive split-by-comma that would break on quoted text containing commas.

Most "AI detectors" are thin wrappers around third-party APIs, giving users no visibility into what signals are being used or how the classification works. Building an interpretable classifier requires extracting meaningful features from raw text signals that capture the structural, lexical, and discourse-level patterns that distinguish AI-generated text from human writing, without relying on a pre-trained language model at inference time.

Designed a 29-feature extraction pipeline in Python covering four feature families: (1) lexical diversity Shannon entropy of the word frequency distribution, MATTR (Moving Average Type-Token Ratio over 20-word windows), lexical density, and top-word frequency ratio; (2) syntactic rule-based POS ratios for verbs, nouns, adjectives, and adverbs plus a passive voice ratio computed by detecting auxiliary + past-participle patterns; (3) structural average sentence length (words and characters), sentence length standard deviation, coefficient of variation, comma ratio, total punctuation ratio, and Yule's K for vocabulary richness; (4) discourse bigram and trigram repetition ratios, top-trigram concentration, stopword ratio, Flesch Reading Ease readability score, and a structural parallelism ratio that fingerprints sentences by their opening word and length bucket. A separate cosine-similarity coherence feature measures mean pairwise cosine similarity between adjacent sentence bag-of-words vectors, capturing the even tone and repetitive phrasing typical of AI text.

Handcrafted features were chosen over transformer embeddings for three reasons: interpretability (each feature maps to a known linguistic property, so false positives can be diagnosed), portability (29 floats require no GPU, no model weights, no runtime dependency the same feature extractor runs in Python during training and TypeScript at inference), and speed (feature extraction for a 25,000-character text completes in under 50ms). The tradeoff is that handcrafted features cannot capture deep semantic patterns that a transformer would the system relies on structural and distributional signals rather than meaning.

The classifier needs to produce a portable model file that can run anywhere no scikit-learn, no TensorFlow.js, no ONNX runtime. The model needs z-score normalization (mean/std computed during training), a weighted sigmoid, and class-weighted binary cross-entropy loss, all implemented from scratch. The inference path must be a simple dot product plus sigmoid, with no graph construction, no session management, and no external model server.

Implemented a complete LogisticRegressionScratch class in Python with z-score scaler, balanced class weight resolution, L2 regularization, and gradient descent trained for 3,000 epochs at lr=0.1 with l2_lambda=0.05 on a 95/5 train/test split. The export function serializes weights (w), bias (b), training mean (mean), and training std (std) into a single JSON file. The inference code mirrors the training exactly: (x - mean) / std, then dot product with w, then bias addition, then sigmoid with clipping at ±500 to prevent overflow.

A from-scratch implementation was necessary because scikit-learn's LogisticRegression uses a different regularization convention (C parameter vs. lambda), a different solver (LBFGS vs. vanilla gradient descent), and different default class weight calculations using the Python model directly would not produce identical results. The z-score normalization parameters must be serialized alongside the weights so that inference applies the exact same transformation as the training pipeline. Exporting a raw weight vector plus normalization stats is the simplest possible inference code path: 29 multiplications, 29 additions, one sigmoid no branching, no loops with conditionals, no memory allocation beyond the feature array.

The Python feature extraction pipeline uses string.punctuation (a stdlib constant), numpy for mean/std, sklearn for CountVectorizer and cosine similarity, and the wordfreq library for Zipf frequency tables. None of these exist in a lightweight TypeScript runtime. A naive port would miss subtle differences in sentence splitting, syllable counting, or Yule's K rounding that would cause the inference features to disagree with the training features, silently degrading accuracy.

Built a zero-dependency TypeScript feature extractor (~250 lines) that mirrors the Python pipeline exactly: the same regex for sentence splitting (lookbehind for [.!?] followed by whitespace + uppercase), the same vowel-counting syllable counter with trailing-e adjustment, the same Yule's K formula using raw frequency counts, the same MATTR sliding-window computation. Cosine similarity between adjacent sentences was reimplemented as a manual dot-product-over-norms function using a sparse vocabulary index, replacing sklearn's CountVectorizer. The sentenceSimilarityFeatures function builds a vocab index from all sentences, constructs dense count vectors, and computes pairwise cosine similarity matching the sklearn output for identical inputs. Feature output was verified to match between Python and TypeScript for the same input text.

An alternative was to ship the Python feature code via Pyodide (Python-in-WASM), but this adds ~10MB to the deployment bundle and adds 500ms+ to cold start unacceptable for a latency-sensitive API. The manual TypeScript port was tedious but produces identical results with zero runtime overhead. The cosine similarity function avoids constructing a full dense matrix (which would be O(n² × vocab_size) in memory) by computing similarity between consecutive sentence pairs only, which is O(n × vocab_size) sufficient because AI text typically shows high similarity between adjacent sentences rather than distant ones.