AI / NLP

What I have been learning & working on

Notes from studying AI / NLP — how I read and reproduce papers, the core topics I have worked through, and the techniques I have used in my own research.

How I approach a paper

Start from the exact data the paper uses; if no official code exists, borrow the data-loading and crawling scheme from other reimplementations (the most time-consuming, error-prone part).
Reimplement the model end to end, focusing on the logic rather than copying.
While (and after) implementing, reason about what the model means — both mathematically and intuitively.
Vary the data by dimension to understand how preprocessing interprets dimensionality.
Distill the core so it can be re-applied to problems elsewhere.

Topics studied

NLP data preprocessing: per-language corpus separation, tokenization strategy, vocabulary construction and augmentation
Tokenizers and how vocabularies are built from raw corpora by frequency
Hugging Face Transformers for loading SOTA models across text, vision, audio, and multimodal tasks
Attention for multimodal fusion — mitigating negative information transfer between noisy modalities
NLP evaluation metrics and reproducible training setups

AI Skillset

Deep-learning frameworks

PyTorchHugging Face Transformers

Model families

LLMVLM (vision-language)DDPM / diffusion

Retrieval & RAG — from SHRAG

Retrieval-Augmented Generation (RAG)LLM-as-Query-StrategistBoolean / logical retrievalMultilingual query expansionMultilingual embeddings & cross-lingual QA

Digital twin — from CIP paper

Digital-twin agentsCIP-concept modeling

Classical NLP

BERTTransformerLSTMsoynlp