AI With Rohan
Home
Blog
About
AI With Rohan
Parallelism in AI - Part 3: Tensor Parallelism
This is Part 3 of the
Parallelism in AI
series. In Part 1, we covered Data Parallelism & FSDP, and in Part 2, we explored Pipeline Parallelism. Now, we dive into Tensor…
Mar 11, 2026
Parallelism in AI - Part 2: Pipeline Parallelism
In Part 1, we saw how Data Parallelism replicates the entire model across devices and splits the data. But what happens when a model is too large to fit on a single device…
Mar 8, 2026
Parallelism in AI - Part 1: Data Parallelism & FSDP
One of the core problems with current neural architecture based AI is that it needs a massive amount of computation during training and inference. To perform these…
Feb 13, 2026
Welcome to AI With Rohan
general
My first blog post introducing the site
Sep 13, 2024
No matching items