Samsung's Processing-in-Memory (PIM)(chipsandcheese.com)
278 points by ingve 57 days ago | 106 comments
tl;dr: Samsung's LPDDR5X-PIM embeds MAC units at each of 16 DRAM banks, exploiting internal bandwidth (614 GB/s vs 76.8 GB/s external) for low-precision ML compute, while cleverly staying within the standard LPDDR5X protocol by repurposing special row addresses as mode-switching MMIO. However, integration with real systems is painful: PIM mode changes the semantics of read/write commands, breaking caching, prefetching, and speculative execution, and forcing memory regions to be uncacheable and effectively single-threaded. The author argues practical adoption would require extending DRAM interfaces with dedicated compute commands and cache-coherent memory controllers rather than the current mode-hijacking approach.
HN Discussion:
  • ↑PIM is too constraining for general applications and better suited to ASICs for specific domains
  • •PIM/dataflow architectures are the inevitable long-term future for AI compute
  • ↑Skepticism that this specific implementation will succeed, like many exotic accelerators before it
  • ↑This implementation is flawed because matrix multiplication requires extensive data movement PIM doesn't solve
  • ↑Rather than patching around limitations, the entire computer architecture should be redesigned