AI With Rohan
Home
Blog
About
Blog
Categories
All
(4)
general
(1)
Order By
Default
Title
Date - Oldest
Date - Newest
Parallelism in AI - Part 3: Tensor Parallelism
This is Part 3 of the
Parallelism in AI
series. In Part 1, we covered Data Parallelism & FSDP, and in Part 2, we explored Pipeline Parallelism. Now, we dive into Tensor…
Mar 11, 2026
7 min
Parallelism in AI - Part 2: Pipeline Parallelism
In Part 1, we saw how Data Parallelism replicates the entire model across devices and splits the data. But what happens when a model is too large to fit on a single device…
Mar 8, 2026
6 min
Parallelism in AI - Part 1: Data Parallelism & FSDP
One of the core problems with current neural architecture based AI is that it needs a massive amount of computation during training and inference. To perform these…
Feb 13, 2026
6 min
Welcome to AI With Rohan
general
My first blog post introducing the site
Sep 13, 2024
1 min
No matching items